Easiest Way to Retrieve Twitter Timeline and Hashtags (Twitter OAuth API 1.1)

Written by Kevin Liew on 19 Mar 2013
221,883 Views • Tutorials

Introduction

Twitter terminated its old API, and all of my Twitter tutorials have stopped working! So, here is a follow up to show you how easy is it to retrieve user timeline and hashtag with Twitter REST API 1.1. Of course, I don't just stop there. I integrated my previously written script and modified it to work with Grid-A-Licious plugin to create something that's similar with Pinterest.

Before we start, we need to have all the ingredients ready.

  1. You need to create an application. https://dev.twitter.com/apps.
  2. Once you've done that, you need to get your consumer key, consumer secret, access token and access token secret.
  3. For security reason, Twitter OAuth is done through server side, in this tutorial, we're using PHP. To keep things simple, we are using Twitter Library called CodeBird-PHP.
  4. To make all the tweets we retrieve pretty, we will be using Grid-A-Licious

Once you have all that, we are good to good. This tutorial is a modified version of my previous old Twitter API - Create a Twitter Feed With Hash Tag And Cache Support. However, we won't implement cache control here just to make it easier to understand.

HTML

Alright, first stop. Since all HTML Markup will be generated by jQuery, we only need a div with an id called #jstwitter

<div id="jstwitter"></div>
<div class="item">
{IMG}
    <div class="tweet-wrapper">
        <span class="text">{TEXT}</span>
        <span class="time">
            <a href="{URL}" target="_blank">{AGO}</a>
        </span>
        by 
        <span class="user">{USER}</span>
    </div>
</div>

CSS

I keep CSS really simple, because in the end, I don't think you will be using mine, you probably will style it up according to your website design. So, I spice thing up a little bit. I'm going to create Pinterest style with Grid-A-Licious.

#jstwitter {
    position: relative;
}
#jstwitter .item {
    -webkit-border-radius:5px;
    -moz-border-radius:5px;
    border-radius:5px;
    -webkit-box-shadow:0 0 3px 1px rgba(100,100,100,0.2);
    -moz-box-shadow:0 0 3px 1px rgba(100,100,100,0.2);
    box-shadow:0 0 3px 1px rgba(100,100,100,0.2);
    overflow:hidden;
    background: #fff;
}
#jstwitter .tweet-wrapper {
    padding:10px;
    -webkit-box-sizing:border-box;
    -moz-box-sizing:border-box;
    box-sizing:border-box;
    line-height:16px;
}
#jstwitter .item a {
    text-decoration: none;
    color: #03a8e5;
}
#jstwitter .item img {
    width:100%;
}
#jstwitter .item a:hover {
    text-decoration: underline;
}
#jstwitter .item .text {
    display:block;
}
#jstwitter .item .time, #jstwitter .tweet .user {
    font-style: italic;
    color: #666666;    
}

PHP

I keep it really simple. We're not going to write our own PHP OAuth authentication, we will be using a Twitter Library called CodeBird-PHP. CodeBird makes it ridiculously easy to get authenticated. You may want to read more about it - CodeBird Documentation. As long as you get all the keys, tokens and secrets right, you should have any problems at all.

I have write inline comment in the following PHP script. If you want to add cache control to beat the rate limits in Twitter API. Here is the place you need to modify it.

<?
//We use already made Twitter OAuth library
//https://github.com/mynetx/codebird-php
require_once ('codebird.php');

//Twitter OAuth Settings
$CONSUMER_KEY = '...';
$CONSUMER_SECRET = '...';
$ACCESS_TOKEN = '...';
$ACCESS_TOKEN_SECRET = '...';

//Get authenticated
Codebird::setConsumerKey($CONSUMER_KEY, $CONSUMER_SECRET);
$cb = Codebird::getInstance();
$cb->setToken($ACCESS_TOKEN, $ACCESS_TOKEN_SECRET);

//retrieve posts
$q = $_POST['q'];
$count = $_POST['count'];
$api = $_POST['api'];

//https://dev.twitter.com/docs/api/1.1/get/statuses/user_timeline
//https://dev.twitter.com/docs/api/1.1/get/search/tweets
$params = array(
'screen_name' => $q,
'q' => $q,
'count' => $count
);

//Make the REST call
$data = (array) $cb->$api($params);

//Output result in JSON, getting it ready for jQuery to process
echo json_encode($data);
?>

Javascript / jQuery

Lastly, the Javascript. We called the PHP we just created. Depend on the parameters, I have made it so you can either search for hashtag, or load a user timeline. It send the parameters to the PHP file above, and PHP script get us authenticated, and PHP returns Twitter data in JSON format. The following scripts read the data and parse them in to HTML markup.

For more information about the Twitter Object, you can read it here.

$(function() {		
			
JQTWEET = {
     
    // Set twitter hash/user, number of tweets & id/class to append tweets
    // You need to clear tweet-date.txt before toggle between hash and user
    // for multiple hashtags, you can separate the hashtag with OR, eg:
    // hash: '%23jquery OR %23css'			    
    search: '%23heroes2013', //leave this blank if you want to show user's tweet
    user: 'quenesstestacc', //username
    numTweets: 21, //number of tweets
    appendTo: '#jstwitter',
    useGridalicious: true,
    template: '<div class="item">{IMG}<div class="tweet-wrapper"><span class="text">{TEXT}</span>\
               <span class="time"><a href="{URL}" target="_blank">{AGO}</a></span>\
               by <span class="user">{USER}</span></div></div>',
     
    // core function of jqtweet
    // https://dev.twitter.com/docs/using-search
    loadTweets: function() {

        var request;
         
        // different JSON request {hash|user}
        if (JQTWEET.search) {
          request = {
              q: JQTWEET.search,
              count: JQTWEET.numTweets,
              api: 'search_tweets'
          }
        } else {
          request = {
              q: JQTWEET.user,
              count: JQTWEET.numTweets,
              api: 'statuses_userTimeline'
          }
        }

        $.ajax({
            url: 'grabtweets.php',
            type: 'POST',
            dataType: 'json',
            data: request,
            success: function(data, textStatus, xhr) {
	            
	            if (data.httpstatus == 200) {
	            	if (JQTWEET.search) data = data.statuses;

                var text, name, img;	         
                	                
                try {
                  // append tweets into page
                  for (var i = 0; i < JQTWEET.numTweets; i++) {		
                  
                    img = '';
                    url = 'http://twitter.com/' + data[i].user.screen_name + '/status/' + data[i].id_str;
                    try {
                      if (data[i].entities['media']) {
                        img = '<a href="' + url + '" target="_blank"><img data-src="' + data[i].entities['media'][0].media_url + '" /></a>';
                      }
                    } catch (e) {  
                      //no media
                    }
                  
                    $(JQTWEET.appendTo).append( JQTWEET.template.replace('{TEXT}', JQTWEET.ify.clean(data[i].text) )
                        .replace('{USER}', data[i].user.screen_name)
                        .replace('{IMG}', img)                                
                        .replace('{AGO}', JQTWEET.timeAgo(data[i].created_at) )
                        .replace('{URL}', url )			                            
                        );
                  }
                
                } catch (e) {
                  //item is less than item count
                }
                
	             if (JQTWEET.useGridalicious) {                
	                //run grid-a-licious
			$(JQTWEET.appendTo).gridalicious({
				gutter: 13, 
				width: 200, 
				animate: true
			});	                   
		     }                  
                    
               } else alert('no data returned');
             
            }   
 
        });
 
    }, 
     
         
    /**
      * relative time calculator FROM TWITTER
      * @param {string} twitter date string returned from Twitter API
      * @return {string} relative time like "2 minutes ago"
      */
    timeAgo: function(dateString) {
        var rightNow = new Date();
        var then = new Date(dateString);
         
        if ($.browser.msie) {
            // IE can't parse these crazy Ruby dates
            then = Date.parse(dateString.replace(/( \+)/, ' UTC$1'));
        }
 
        var diff = rightNow - then;
 
        var second = 1000,
        minute = second * 60,
        hour = minute * 60,
        day = hour * 24,
        week = day * 7;
 
        if (isNaN(diff) || diff < 0) {
            return ""; // return blank string if unknown
        }
 
        if (diff < second * 2) {
            // within 2 seconds
            return "right now";
        }
 
        if (diff < minute) {
            return Math.floor(diff / second) + " seconds ago";
        }
 
        if (diff < minute * 2) {
            return "about 1 minute ago";
        }
 
        if (diff < hour) {
            return Math.floor(diff / minute) + " minutes ago";
        }
 
        if (diff < hour * 2) {
            return "about 1 hour ago";
        }
 
        if (diff < day) {
            return  Math.floor(diff / hour) + " hours ago";
        }
 
        if (diff > day && diff < day * 2) {
            return "yesterday";
        }
 
        if (diff < day * 365) {
            return Math.floor(diff / day) + " days ago";
        }
 
        else {
            return "over a year ago";
        }
    }, // timeAgo()
     
     
    /**
      * The Twitalinkahashifyer!
      * http://www.dustindiaz.com/basement/ify.html
      * Eg:
      * ify.clean('your tweet text');
      */
    ify:  {
      link: function(tweet) {
        return tweet.replace(/\b(((https*\:\/\/)|www\.)[^\"\']+?)(([!?,.\)]+)?(\s|$))/g, function(link, m1, m2, m3, m4) {
          var http = m2.match(/w/) ? 'http://' : '';
          return '<a class="twtr-hyperlink" target="_blank" href="' + http + m1 + '">' + ((m1.length > 25) ? m1.substr(0, 24) + '...' : m1) + '</a>' + m4;
        });
      },
 
      at: function(tweet) {
        return tweet.replace(/\B[@ï¼ ]([a-zA-Z0-9_]{1,20})/g, function(m, username) {
          return '<a target="_blank" class="twtr-atreply" href="http://twitter.com/intent/user?screen_name=' + username + '">@' + username + '</a>';
        });
      },
 
      list: function(tweet) {
        return tweet.replace(/\B[@ï¼ ]([a-zA-Z0-9_]{1,20}\/\w+)/g, function(m, userlist) {
          return '<a target="_blank" class="twtr-atreply" href="http://twitter.com/' + userlist + '">@' + userlist + '</a>';
        });
      },
 
      hash: function(tweet) {
        return tweet.replace(/(^|\s+)#(\w+)/gi, function(m, before, hash) {
          return before + '<a target="_blank" class="twtr-hashtag" href="http://twitter.com/search?q=%23' + hash + '">#' + hash + '</a>';
        });
      },
 
      clean: function(tweet) {
        return this.hash(this.at(this.list(this.link(tweet))));
      }
    } // ify
 
     
};		

});

And finally, you run the script like this:

$(function () {
    // start jqtweet!
    JQTWEET.loadTweets();
});

Conclusion

That's it my friend. If you have knowledge in both backend and frontend, web development can be easier. We can reuse a lot of scripts, for example, we use CodeBird, Gridalicious, my previous Twitter scripts. What we did right here, we use it, modify it, and bang, we have a working beautiful twitter feed interface.

I hope you will find this tutorial useful, if you have any question, don't hesitate to drop us a comment. :)

Demo Download
Join the discussion

Comments will be moderated and rel="nofollow" will be added to all links. You can wrap your coding with [code][/code] to make use of built-in syntax highlighter.

185 comments
Jeroen 12 years ago
Hello,

I find it quite difficult to install this tutorial. What is the structure of the maps and which files do I need? Before the v1.1 api I used your tutorial which worked, now I cannot install it anymore, its quite difficult to follow, with the download files and the custom scripts.

What is the mapstructure to upload and where do i have to customize data? Only the keys and my accountname from twitter?

Greets,

jeroen
Reply
Kevin Admin 12 years ago
Hi Jeroen, Oauth is bit complicated.

1. First, you need to get your Twitter API key - https://dev.twitter.com/apps . After that, fill in the detail in grabtweets.php

2. And then, edit the Javascript file jquery.jstwitter.js - either retrieve tweet based on a specific user or search hashtag.

That's it.

Reply
rinaldi 12 years ago
Hi kevin,

i wanna ask you how to get all tweet from hashtag without username like topsy and how to count the reply tweet. cus when i used your script not all tweet can viewed.

thanks
Reply
Kevin Admin 12 years ago
You can search hashtag, just leave username blank.
Reply
John Mathew 12 years ago
Is it possible to combine the Instagram api with this to display a mix of Instagram and twitter hashtag results?
Reply
Boykov 12 years ago
Hi,
I had rigged it and it worked perfectly until a few weeks ago. Since then I get no output in browser. grabtweets request returns {"statuses":[],"search_metadata":{"completed_in":0.007,"max_id":375272748475367424,"max_id_str":"375272748475367424","query":"xx","refresh_url":"?since_id=375272748475367424&q=xx&include_entities=1","count":100,"since_id":0,"since_id_str":"0"},"httpstatus":200}

How do I fix it?
Reply
designboy 12 years ago
Hi,

Is there a way of only showing the tweets that have images in them?

Many Thanks
Reply
takeinpixels 12 years ago
Is there a way to remove re-tweets? many thanks
Reply
@SugarPillStdios 11 years ago
Adding the "'exclude' => 'retweets'" line to your grab retweets.php as follows, seems to accomplish this for me.

$params = array(
'screen_name' => $q,
'q' => $q,
'count' => $count,
'exclude' => 'retweets'
);
Reply
Jeroen 12 years ago
This is what i got so far, and grabtweets.php, can't find that file anywhere..

Im doing something horribly wrong.


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Feed test 4</title>
<link href="css.css" rel="stylesheet" type="text/css" />

</head>

<body>

<div id="jstwitter"></div>
<div class="item">
{IMG}
<div class="tweet-wrapper">
<span class="text">{TEXT}</span>
<span class="time">
<a href="{URL}" target="_blank">{AGO}</a>
</span>
by
<span class="user">{USER}</span>
</div>
</div>
<script>
$(function () {
// start jqtweet!
JQTWEET.loadTweets();
});
</script>

<?
//We use already made Twitter OAuth library
//http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
require_once ('codebird.php');

//Twitter OAuth Settings
CENS

//Get authenticated
Codebird::setConsumerKey($CONSUMER_KEY, $CONSUMER_SECRET);
$cb = Codebird::getInstance();
$cb->setToken($ACCESS_TOKEN, $ACCESS_TOKEN_SECRET);

//retrieve posts
$q = $_POST['q'];
$count = $_POST['count'];
$api = $_POST['api'];

//http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
//http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
$params = array(
'screen_name' => $q,
'q' => $q,
'count' => $count
);

//Make the REST call
$data = (array) $cb->$api($params);

//Output result in JSON, getting it ready for jQuery to process
echo json_encode($data);
?>

</body>

<script>
$(function() {

JQTWEET = {

// Set twitter hash/user, number of tweets & id/class to append tweets
// You need to clear tweet-date.txt before toggle between hash and user
// for multiple hashtags, you can separate the hashtag with OR, eg:
// hash: '%23jquery OR %23css'
search: '%23heroes2013', //leave this blank if you want to show user's tweet
user: 'quenesstestacc', //username
numTweets: 21, //number of tweets
appendTo: '#jstwitter',
useGridalicious: true,
template: '<div class="item">{IMG}<div class="tweet-wrapper"><span class="text">{TEXT}</span>\
<span class="time"><a href="{URL}" target="_blank">{AGO}</a></span>\
by <span class="user">{USER}</span></div></div>',

// core function of jqtweet
// http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
loadTweets: function() {

var request;

// different JSON request {hash|user}
if (JQTWEET.search) {
request = {
q: JQTWEET.search,
count: JQTWEET.numTweets,
api: 'search_tweets'
}
} else {
request = {
q: JQTWEET.user,
count: JQTWEET.numTweets,
api: 'statuses_userTimeline'
}
}

$.ajax({
url: 'grabtweets.php',
type: 'POST',
dataType: 'json',
data: request,
success: function(data, textStatus, xhr) {

if (data.httpstatus == 200) {
if (JQTWEET.search) data = data.statuses;

var text, name, img;

try {
// append tweets into page
for (var i = 0; i < JQTWEET.numTweets; i++) {

img = '';
url = 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> + data[i].user.screen_name + '/status/' + data[i].id_str;
try {
if (data[i].entities['media']) {
img = '<a href="' + url + '" target="_blank"><img src="' + data[i].entities['media'][0].media_url + '" /></a>';
}
} catch (e) {
//no media
}

$(JQTWEET.appendTo).append( JQTWEET.template.replace('{TEXT}', JQTWEET.ify.clean(data[i].text) )
.replace('{USER}', data[i].user.screen_name)
.replace('{IMG}', img)
.replace('{AGO}', JQTWEET.timeAgo(data[i].created_at) )
.replace('{URL}', url )
);
}

} catch (e) {
//item is less than item count
}

if (JQTWEET.useGridalicious) {
//run grid-a-licious
$(JQTWEET.appendTo).gridalicious({
gutter: 13,
width: 200,
animate: true
});
}

} else alert('no data returned');

}

});

},


/**
* relative time calculator FROM TWITTER
* @param {string} twitter date string returned from Twitter API
* @return {string} relative time like "2 minutes ago"
*/
timeAgo: function(dateString) {
var rightNow = new Date();
var then = new Date(dateString);

if ($.browser.msie) {
// IE can't parse these crazy Ruby dates
then = Date.parse(dateString.replace(/( \+)/, ' UTC$1'));
}

var diff = rightNow - then;

var second = 1000,
minute = second * 60,
hour = minute * 60,
day = hour * 24,
week = day * 7;

if (isNaN(diff) || diff < 0) {
return ""; // return blank string if unknown
}

if (diff < second * 2) {
// within 2 seconds
return "right now";
}

if (diff < minute) {
return Math.floor(diff / second) + " seconds ago";
}

if (diff < minute * 2) {
return "about 1 minute ago";
}

if (diff < hour) {
return Math.floor(diff / minute) + " minutes ago";
}

if (diff < hour * 2) {
return "about 1 hour ago";
}

if (diff < day) {
return Math.floor(diff / hour) + " hours ago";
}

if (diff > day && diff < day * 2) {
return "yesterday";
}

if (diff < day * 365) {
return Math.floor(diff / day) + " days ago";
}

else {
return "over a year ago";
}
}, // timeAgo()


/**
* The Twitalinkahashifyer!
* http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
* Eg:
* ify.clean('your tweet text');
*/
ify: {
link: function(tweet) {
return tweet.replace(/\b(((https*\:\/\/)|www\.)[^\"\']+?)(([!?,.\)]+)?(\s|$))/g, function(link, m1, m2, m3, m4) {
var http = m2.match(/w/) ? 'http://' : '';
return '<a class="twtr-hyperlink" target="_blank" href="' + http + m1 + '">' + ((m1.length > 25) ? m1.substr(0, 24) + '...' : m1) + '</a>' + m4;
});
},

at: function(tweet) {
return tweet.replace(/\B[@ï¼ ]([a-zA-Z0-9_]{1,20})/g, function(m, username) {
return '<a target="_blank" class="twtr-atreply" href="http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> + username + '">@' + username + '</a>';
});
},

list: function(tweet) {
return tweet.replace(/\B[@ï¼ ]([a-zA-Z0-9_]{1,20}\/\w+)/g, function(m, userlist) {
return '<a target="_blank" class="twtr-atreply" href="http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> + userlist + '">@' + userlist + '</a>';
});
},

hash: function(tweet) {
return tweet.replace(/(^|\s+)#(\w+)/gi, function(m, before, hash) {
return before + '<a target="_blank" class="twtr-hashtag" href="http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> + hash + '">#' + hash + '</a>';
});
},

clean: function(tweet) {
return this.hash(this.at(this.list(this.link(tweet))));
}
} // ify


};

});


</script>

</html>
Reply
Kevin Admin 12 years ago
Please download the demo file.
Reply
Miguel Oliveira 12 years ago
Hi,
Can this display Tweets with an Hashtag only from a given user?
Reply
Tom 12 years ago
Hey, Thanks for the tutorial. I'm getting a JS error (popup) - "No Data Returned". When i Try debugging the grabtweets.php file it's giving me an error:
Method name must be a string... On line 33.

Any ideas on how to fix this? I tried outputting the $api variable but it's returning NULL..

Thanks in advance
Reply
Kevin Admin 12 years ago
Not sure what's going on. It could be PHP version. You can read more about it in previous comments. Or try this.


Codebird\Codebird::setConsumerKey($CONSUMER_KEY, $CONSUMER_SECRET);
$cb = Codebird\Codebird::getInstance();
Reply
lennyD 12 years ago
Hi,

I am having a problem on our server specific. I get this error.


POST our_url/grabtweets.php 500 (Internal Server Error) jquery.js:8724
send jquery.js:8724
x.extend.ajax jquery.js:8154
JQTWEET.loadTweets jquery.jstwitter.js?ver=1.0.0:39
(anonymous function) scripts.js?ver=1.0.0:117
c jquery.js:3074
p.fireWith jquery.js:3186
x.extend.ready jquery.js:433
q


I can't find the solution and I have no problems ons other servers. Do you have any idea what this could be?
Reply
Adam 12 years ago
Hey Kevin,
Where can i set the amount of tweets? i only get around 78 even though i have the numTweets var set to 200. Am i missing something? Is there a limit set by codebird?
Thanks
Adam
Reply
Kevin Admin 12 years ago
It shouldn't. Max tweet returned for this API is 3200 tweets.

So, I'm not so sure what's going on here.
Reply
Adam 12 years ago
yeah not sure either I can send you my HTML/PHP if you want? I can not post it up here since its for a project.
not sure if its just because its

window.onload =function(){
var myHashtag = '<?php echo($hash);?>';
JQTWEET.search = myHashtag;
JQTWEET.numTweets = 500;
JQTWEET.appendTo = '#jstwitter';
JQTWEET.loadTweets();

}

Reply
jackie 12 years ago
hello! thanks in advance for the help - I can get the tweets from the user just fine - its when I add the additional search for the hashtag in that I get zero results returned even though I know the hashtag is correct and matches with items in the twitter stream. Any ideas? Thank you!
Reply
Tatters 12 years ago
Hi Kevin,

I've updated the JS code so it looks at a text area for a search term like so:

search: $('textarea[name=search]').val(),

The HTML and JS to call it looks like so:

<textarea rows="5" name="search" type="text" id="term">%23NFL</textarea>

<script>
$(document).ready(function () {
$('#button').click(JQTWEET.loadTweets);
});
</script>

The one problem is I have to hard code the search term in the text area html (e.g. %23NFL) to call the JS.

Do you know how to update the JS so it pulls from a search term typed into the textarea? I've been scratching my head for a week now.

Thanks for the help in advance.
Reply
Peter 12 years ago
So I've been trying to get this to work.. I ran through most of the same problems people were having.. and I cancelled out all errors and now I'm not getting anything showing up.

Do you know of any other reasons why this may not work? I am testing this locally using MAMP Pro would that be a reason for not pulling in any data?

Thanks for the tut!

P
Reply
Kevin Liew Admin 12 years ago
There is a chance that your PHP version is not supported. Can you go to CodeBird for PHP version and also your own server?
Reply
sumit 12 years ago
hello Kevin Liew,
thanks this code .but please solve my problem .in this code i want create text area .and inter twitter name after show all twitte's please help .how to pass value index file to jquery.jstwitter.js
user: 'quenesstestacc', //username
Reply