May 28, 2016

One of the reasons why Twitter has become so popular is because it is a great way to get data for web applications and projects. Through its Search API, it is possible to find content by issuing a query to Twitter based on a supplied string. The results can then be parsed or displayed as preferred using ancillary tools.

The purpose of this tutorial is that you learn how to create an interactive web app that retrieves geolocated tweets and shows them in a map. Sounds cool? For facilitating the app creation, we are going to use Shiny, a web application framework for R. Let’s take a look at how we can easily do that.


Get access to Twitter API

To get started, create a Twitter account. Once signed up, go to apps.twitter.com and create a new app in order to get access to the Twitter API (short for Application Programming Interface). In the ‘Keys and Access Tokens’ section of the new app you’ll be given the following: Consumer Key, Consumer Secret, Access Token, and Access Token Secret.

Notice that there are some rate limits for the Twitter Search API that you should take into account. Also note that only a small percentage of public tweets are geolocated (about 3-5%) so it is possible to not retrieve as many tweets as requested.

Prepare the code for your app

First open an R session (preferably in RStudio) and install the following packages: twitteR, shiny and leaflet. Then create a new R script. For keeping it simple, we are going to put all the necessary R code in just one file that we are going to name app.R.

A Shiny app has two components. The first part, called ui, defines the visual appearance or user-interface of the app. Let’s take a look at the code:

library(leaflet)
 library(twitteR)

shinyApp(
  ui = fluidPage(
    fluidRow(
      column(4, textInput("searchkw", label = "search:", value = "#movie")),
      column(4, textInput("lat", label = "latitude:", value = 40.75)),
      column(4, textInput("long", label = "longitude:", value = -74)),
      column(8, leafletOutput("myMap")),
      column(12, tableOutput('table'))
    )
  ),


fluidPage and fluidRow are used for creating a fluid page layout. We are going to include three text input controls: one for a search keyword and two other for lat/long coordinates. Additionaly, our app will include two outputs: a leaflet map and a table with the resulting tweets.

The second component of a Shiny app is the server which contains the instructions for building the app. Let’s examine the code:

  server = function(input, output) {
    
    # OAuth authentication
    consumer_key <- readLines("tokens.txt")[1]
    consumer_secret <- readLines("tokens.txt")[2]
    access_token <- readLines("tokens.txt")[3]
    access_secret <- readLines("tokens.txt")[4]
    options(httr_oauth_cache = TRUE) # enable using a local file to cache OAuth access credentials between R sessions
    setup_twitter_oauth(consumer_key, consumer_secret, access_token, access_secret)
    
    # Issue search query to Twitter
    dataInput <- reactive({  
      tweets <- twListToDF(searchTwitter(input$searchkw, n = 100, 
                                        geocode = paste0(input$lat, ",", input$long, ",10km"))) 
      tweets$created <- as.character(tweets$created)
      tweets <- tweets[!is.na(tweets[, "longitude"]), ]
    })
    
    # Create a reactive leaflet map
    mapTweets <- reactive({
      map = leaflet() %>% addTiles() %>%
        addMarkers(as.numeric(dataInput()$longitude), as.numeric(dataInput()$latitude), popup = dataInput()$screenName) %>%
        setView(input$long, input$lat, zoom = 11)
    })
    output$myMap = renderLeaflet(mapTweets())
    
    # Create a reactive table 
    output$table <- renderTable(
      dataInput()[, c("text", "screenName", "longitude", "latitude", "created")]
    )
  }
)


First, all the four authentication parameters (consumer key, consumer secret, access token and access secret) are read from a text file and supplied to the OAuth authentication. I named the text file ‘tokens.txt’ and saved it in the same folder where the app.R file is stored (Note: if you get a warning when reading the text file (‘Warning in readLines(“tokens.txt”) : incomplete final line found on ‘tokens.txt’‘), then add an empty fifth line to the file).

Then comes the main part of our app. dataInput is a reactive function that issues a query to Twitter based on a search string (input$searchkw) provided by the user. As we are interested in geocoded tweets, it is necessary to provide search radius and latitude/longitude coordinates in the format latitude,longitude,radius. In this sample app, the maximum number of tweets to return (n = 100) and the search radius (10km) are fixed for simplicity, but this could be easily modified, so feel free to experiment. Notice the last line of dataInput: !is.na() is used to remove those tweets that are not geocoded.

In the final part, a leaflet map and a table are created based on query results stored in dataInput. Popups in the leaflet map show the screen name (i.e., twitter username) while the output table contains the tweet, the location and the date the tweet was created, besides the username. For a more detailed explanation about the creation of a leaflet map using R see this post.

Want to test the app? Just click the ‘Run App’ button in RStudio. You should see something like this:

Testing the app in RStudio


Deploy/Share your app

The easiest way for sharing your Shiny apps online is to use shinyapps.io by RStudio (alternatively, you can use Shiny Server if you own a server or have access to one). With shinyapps.io, you can upload your app directly from an R session to an RStudio server.

In short, you need to create a shinyapps.io account and then install and configure an R package called rsconnect. The complete set of instructions is provided in this excellent tutorial so I’m not going into details in this post.

With your shinyapps.io account correctly configured, you’ll be able to deploy your app by clicking the ‘Publish’ button in the RStudio IDE, or by executing the following command lines:

library(rsconnect)
 deployApp()


After deployment, the app is automatically launched and opened in your web browser pointing to a web address similar to “https://your-account-name.shinyapps.io/your-app-name/”.

That’s basically all you need to do to create a working web application. Watch the following video to see the whole process for creating and deploying this Shiny tweet-mapping app:


Further steps

For this tutorial I kept the user interface of the app to the minimum. However, it is quite easy to create elegant Shiny apps with some additional work. For inspiration, see the samples and catalogs in this web page. Regarding apps using Twitter data, see this NRL dashboard app. Twitter data can be processed for more advanced analyses, as shown in this example for predicting movie success based on sentiment analysis of tweets.

Hope this post motivates you to create your own app soon!


You may also be interested in:

* Web mapping with Leaflet and R


Share this post:   



Subscribe to my blog and get the '50 best QGIS plugins of 2016' ebook completely free!