Create a Twitter Feed With Hash Tag And Cache Support

Written by Kevin Liew on 20 Feb 2012
59,583 Views • Tutorials

Introduction

Due to popular demand, this tutorial is actually a revisit from my previous Twitter tutorial - Create a dead simple Twitter feed with jQuery. A lot of readers requested to be able to parse hashtag. So, I've decided to make a new version that able to do both plus some enhancements.

The foundation of this script will be the same, but with some modification to accept both hashtag and normal user feed. It will be smart enough to switch but with one tiny caveat which I will mention later on. You can see the preview or download it to play with the code.

UPDATE: We have written an updated version of Twitter API tutorial based on Twitter Newest API 1.1. - Easiest Way to Retrieve Twitter Timeline and Hashtags

Features

  • Able to parse User Timeline and hashtag (multiple hashtags by using OR operator to separate hashtags)
  • Cache result to avoid rate limits restriction from Twitter.
  • Decided to not using Cronjob to renew the cache, it uses date comparison with PHP.
  • Using links/hashtag/alias script to parse the tweet.

HTML

HTML is basically the same as the old one, nothing fancy.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
    <meta http-equiv="Content-Type" content="text/html;charset=UTF-8" />
    <title></title>
    <script data-src="http://ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.min.js" type="text/javascript"></script>
    <script data-src="twitter.js"></script>
</head>
<body>
     
    <div id="jstwitter">
    </div>
     
</body>
</html>

CSS

CSS, nothing special, just simply style it up to make it presentable. However, this is a list of class that you can use to style it up:

  • .twtr-hashtag: #abc
  • .twtr-hyperlink: hyperlink
  • .twtr-atreply: #abc
  • .text: tweet
  • .user: username
  • .time: relative time (10 minutes ago)
body {
	background:#ccc;
}

#jstwitter {
	width: 300px;
	font-family: georgia;
	font-size: 12px;
	color: #333333;
	padding: 10px 10px 0 10px;
	margin:0 auto;
	background:#fff;
	border:5px solid #eaeaea;
	border-radius:15px;
	box-shadow:0 0 5px rgba(0, 0, 0, 0.3);
}

#jstwitter h1 {
	text-align:center;
	margin-top:5px;
}

#jstwitter .tweet {
	margin: 0 auto 15px auto;
	padding: 0 0 15px 0;
	border-bottom: 1px dotted #ccc;
}

#jstwitter .tweet a {
	text-decoration: none;
	color: #03a8e5;
}

#jstwitter .tweet a:hover {
	text-decoration: underline;
}

#jstwitter .tweet .text {
	display:block;
}

#jstwitter .tweet .time, #jstwitter .tweet .user {
	font-style: italic;
	color: #666666;
}

#jstwitter .tweet:last-child {
	border:0;
}

Javascript

Okay, the core. This jQuery script added a few if-else statements to toggle between hashtag and user timeline. I put inline comment in every crucial code for better understanding of what it does.

The way it works:

  1. Firstly, you can set either hashtag or username. If you filled in hashtag, username will be ignore. If you want username, you need to leave hashtag empty.
  2. You can set other setting such as number of tweet, cache expiry date, append to element.
  3. In loadTweet function, it constructs different request and pass it to PHP script, and then PHP script will format the request with the right Twitter API and grab the content. The PHP script is responsible to handle the cache as well. Depend on how long you set in the cacheExpiry, it will renew the cache accordingly.
  4. Finally, the Javascript will parse the returned JSON based on the type of API. (hashtag and user timeline's JSON is a bit different in structure)

Some tips and tricks

Multiple Hashtags: You can separate the hastags with OR operator. eg: '%23jquery OR %23css'

Multiple Users: You can use 'from:account' and separate with OR operator. eg: 'from:quenesswebblog OR from:favbulous'

JQTWEET = {
	
	// Set twitter hash/user, number of tweets & id/class to append tweets
	// You need to clear tweet-date.txt before toggle between hash and user
	// for multiple hashtags, you can separate the hashtag with OR, eg:
	// hash: '%23jquery OR %23css'
	hash: '%23jquery', //leave this blank if you want to show user's tweet
	user: 'quenesswebblog', //username
	numTweets: 5, //number of tweets
	cacheExpiry: 2, //get the new cache in 2 hours
	appendTo: '#jstwitter',
	
	// core function of jqtweet
	// https://dev.twitter.com/docs/using-search
	loadTweets: function() {
	
		var request;
		
		// different JSON request {hash|user}
		if (JQTWEET.hash) {
			request = {
				q: JQTWEET.hash,
				expiry: JQTWEET.cacheExpiry,				
				api: 'http://search.twitter.com/search.json'
			}
		} else {
			request = {
				screen_name: JQTWEET.user,
				include_rts: true,
				count: JQTWEET.numTweets,
				include_entities: true,
				expiry: JQTWEET.cacheExpiry, 
				api: 'http://api.twitter.com/1/statuses/user_timeline.json/'
			}
		}

		$.ajax({
			url: 'tweets-grab.php',
			type: 'GET',
			dataType: 'json',
			data: request,
			success: function(data, textStatus, xhr) {

				var text, name, html = '<div class="tweet"><span class="text">TWEET_TEXT</span><span class="time"><a href="URL" target="_blank">AGO</a></span> by <span class="user">USER</span></div>';
		
				try {
		
					//Twitter Search API has different JSON Structure
					if (JQTWEET.hash) data = data['results'];
		
					// append tweets into page
					for (var i = 0; i < data.length && i < JQTWEET.numTweets; i++) {
						
						name = (JQTWEET.hash) ? data[i].from_user : data[i].user.screen_name;
	
						$(JQTWEET.appendTo).append(	
						    html.replace('TWEET_TEXT', JQTWEET.ify.clean(data[i].text) )
						        .replace(/USER/g, name)
						        .replace('AGO', JQTWEET.timeAgo(data[i].created_at) )
						        .replace('URL', 'http://twitter.com/' + data[i].from_user + '/status/' + data[i].id_str )
						);								
	
					}					
				
				} catch (e) {
					alert('No data returned, you might want to clear tweets-date.txt.');
				}			
			
			}	

		});

	}, 
	
		
	/**
      * relative time calculator FROM TWITTER
      * @param {string} twitter date string returned from Twitter API
      * @return {string} relative time like "2 minutes ago"
      */
    timeAgo: function(dateString) {
		var rightNow = new Date();
		var then = new Date(dateString);
		
		if ($.browser.msie) {
			// IE can't parse these crazy Ruby dates
			then = Date.parse(dateString.replace(/( \+)/, ' UTC$1'));
		}

		var diff = rightNow - then;

		var second = 1000,
		minute = second * 60,
		hour = minute * 60,
		day = hour * 24,
		week = day * 7;

		if (isNaN(diff) || diff < 0) {
			return ""; // return blank string if unknown
		}

		if (diff < second * 2) {
			// within 2 seconds
			return "right now";
		}

		if (diff < minute) {
			return Math.floor(diff / second) + " seconds ago";
		}

		if (diff < minute * 2) {
			return "about 1 minute ago";
		}

		if (diff < hour) {
			return Math.floor(diff / minute) + " minutes ago";
		}

		if (diff < hour * 2) {
			return "about 1 hour ago";
		}

		if (diff < day) {
			return  Math.floor(diff / hour) + " hours ago";
		}

		if (diff > day && diff < day * 2) {
			return "yesterday";
		}

		if (diff < day * 365) {
			return Math.floor(diff / day) + " days ago";
		}

		else {
			return "over a year ago";
		}
	}, // timeAgo()
    
	
    /**
      * The Twitalinkahashifyer!
      * http://www.dustindiaz.com/basement/ify.html
      * Eg:
      * ify.clean('your tweet text');
      */
    ify:  {
      link: function(tweet) {
        return tweet.replace(/\b(((https*\:\/\/)|www\.)[^\"\']+?)(([!?,.\)]+)?(\s|$))/g, function(link, m1, m2, m3, m4) {
          var http = m2.match(/w/) ? 'http://' : '';
          return '<a class="twtr-hyperlink" target="_blank" href="' + http + m1 + '">' + ((m1.length > 25) ? m1.substr(0, 24) + '...' : m1) + '</a>' + m4;
        });
      },

      at: function(tweet) {
        return tweet.replace(/\B[@@]([a-zA-Z0-9_]{1,20})/g, function(m, username) {
          return '<a target="_blank" class="twtr-atreply" href="http://twitter.com/intent/user?screen_name=' + username + '">@' + username + '</a>';
        });
      },

      list: function(tweet) {
        return tweet.replace(/\B[@@]([a-zA-Z0-9_]{1,20}\/\w+)/g, function(m, userlist) {
          return '<a target="_blank" class="twtr-atreply" href="http://twitter.com/' + userlist + '">@' + userlist + '</a>';
        });
      },

      hash: function(tweet) {
        return tweet.replace(/(^|\s+)#(\w+)/gi, function(m, before, hash) {
          return before + '<a target="_blank" class="twtr-hashtag" href="http://twitter.com/search?q=%23' + hash + '">#' + hash + '</a>';
        });
      },

      clean: function(tweet) {
        return this.hash(this.at(this.list(this.link(tweet))));
      }
    } // ify

	
};



$(document).ready(function () {
	// start jqtweet!
	JQTWEET.loadTweets();
});

PHP

Right, the final part of this tutorial - the PHP. It:

  • Checks for Cache Expiry interval
  • Constructs the API call
  • Retrieves JSON data from Twitter
  • Saves both JSON data and date

We need two text files:

  • tweets-cache.txt: contains latest data in JSON format
  • tweets-date.txt: contains the date of last retrieval

In the introduction, I mentioned about a small caveat of this script - Whenever you switch between hashtag and username, you need to clear the tweets-date.txt.

<?php


$cache = 'tweets-cache.txt';
$date = 'tweets-date.txt';

$currentTime = time(); // Current time

// Get cache time
$datefile = fopen($date, 'r');
$cacheDate = fgets($datefile);
fclose($datefile);


//check if cache has expired
if (floor(abs(($currentTime-$cacheDate) / 3600)) <= $_GET['expiry'] && $cacheDate) {

	$cachefile = fopen($cache, 'r');
	$data = fgets($cachefile);
	fclose($cachefile);

} else { //renew the cache

	//toggle between API
	if ($_GET['q']) 
	{
		$data = file_get_contents($_GET['api'] . '?q=' . urlencode($_GET['q']));   
	
	} else if ($_GET['screen_name']) 
	{
		$data = file_get_contents($_GET['api'] . '?screen_name=' . $_GET['screen_name'] . '&count=' . $_GET['count'] . '&include_rts=' . $_GET['include_rts'] . '&include_entities=' . $_GET['include_entities']);   
	}
	
	// update cache file
	$cachefile = fopen($cache, 'wb');  
	fwrite($cachefile,utf8_encode($data));  
	fclose($cachefile); 

	// update date file
	$datefile = fopen($date, 'wb');  
	fwrite($datefile, utf8_encode(time()));  
	fclose($datefile); 	 
}


header('Content-type: application/json');
echo $data;
?>

Conclusion

So, that's how you do it, support both hashtag and username, easy to style and uses cache to overcome Twitter rate limits. You just have to remember, whenever you switch between hashtag and username, remember to clear the content of tweets-date.txt.

That's it, hope you will find this useful and any questions, just drop me a comment and I will try my very best to answer it quickly. Stay tuned with queness! For more cool and updated javascript inspiration, tutorials, plugins, you can follow us on twitter or like us on facebook.

Demo 1Demo 2Download
Join the discussion

Comments will be moderated and rel="nofollow" will be added to all links. You can wrap your coding with [code][/code] to make use of built-in syntax highlighter.

71 comments
azzax 8 years ago
Interestin tutorial. I wanted to know a bit more, as can we add user's pic sidebyside of the Tweets who posted it.
Reply
Kevin Liew Admin 8 years ago
All the data is in the JSON, you can load the profile image via:

data[i].profile_image_url
Reply
ivy 8 years ago
Can i use this to download tweets containing a trending topic in the past? Thank you in advance.
Reply
magus824 8 years ago
This is great stuff and easy to understand, thanks for this! I was just wondering if you could help me with one thing - I want these items to be written directly to the page, so that I can do things with them (like adding tooltips, rollover effects, etc). Right now when I view the source code after the page is loaded, it does not show the tweets, it just shows the container. Any help in this would be appreciated! Thanks!
Reply
Kevin Liew Admin 8 years ago
This script is javascript driven, so you can't really view the source.
Reply
VA 8 years ago
I didn't change any of the downloaded files and uploaded to my server to test and Twitter feed won't display. Just the heading which says #jquery. Hostgator is my host. What am I doing wrong?
Reply
Laura 8 years ago
Same thing here... I haven't changed anything -- just loaded it on my server and it doesn't pull any tweets for me either.
Reply
Kevin Liew Admin 8 years ago
This script involves PHP, does your server has the support? Secondly, you need to make sure you have read write permission on tweets-cache.txt and tweets-date.txt text file.
Reply
Dusan Markovic 8 years ago
doesn't work for me too, the permissions are ok, but it does not populate tweets
Reply
Laura 8 years ago
My server does support PHP, and those two files are readable. http://projects.laurasue.com/macslist/tweet/
Reply
Nate 8 years ago
Laura,
You should make lines 5 and 6 in twitter.js look exactly like this:

hash: '',
user: '%40quenesswebblog',

%40 is the code for @
%23 is the code for # so he originally had hash: '%23jquery', on line 5. You need to leave one or the other blank
Reply
Kevin Liew Admin 8 years ago
Hi Laura, please clear the content of tweets-date.txt
Reply
Tina 8 years ago
I have the same problem. PHP is working, permissions are OK but feed won't show..
Reply
Kevin Liew Admin 8 years ago
Please make sure tweet-date.txt file is empty. Otherwise, please upload it to server, I will have a look.
Reply
Tina 8 years ago
I tried this on another server and it's working (same files). Is it possible that some Apache modules or PHP extensions have to be turned on?
Reply
Gp 8 years ago
I have the same problem. Chrome said: Notice: Undefined index: hours in .. weets-grab.php on line 15. This refers to "$_GET['hours']". Tweets-date.txt is empty. PHP support and permissions are ok. Thanks.
Reply
James 8 years ago
Hi,

Good tutorial, but in your tutorial are the Usernames Hardcoded ? How would you go about getting the User to enter their usernames and then calling it that way ?

Regards
Reply
Albana 8 years ago
Hi, this post is great!

I have been struggling for a couple of days...sorry , i'm new to all this.. i needed to attach images like thumbnail, images, not link, like you have done in another tutorial, can you please show me how to do it?

But be precise.. i'm new to this..
Reply
Kevin Liew Admin 8 years ago
Hi Albana, here is a working example of including media in the feed:
http://www.queness.com/resources/html/jstwitterhashtag/media/

Basically I added:

var img

and

if (data[i].entities.media) {
img = '<img src="' + data[i].entities.media[0].media_url + '" width="100%"/>';
}

You can see it in the working example, just view source and open the js file.
Reply
Albana 8 years ago
Hi Kevin, thanks for the reply..

I have done it before but it won't work with me.. i saw the source code of the file, but i need a hash tag tweet feed, let us say, i need a hashtag of "wearetwo" , i don't need the feeds by user id.. please help me with this... It's not working with me...
Reply
Albana 8 years ago
Can you please tell me if there's something wrong with the code i insert or it's something else? :)

Sorry for asking a lot, it's 2 am here in Albania and i'm getting a hard time to concentrate.. :)
Reply
Albana 8 years ago
Hi Kevin,

sorry for disturbing again... but the code won't work anymore...Sorry for insisting... can you please give me a solution?... I'm a little late with work, and i am stucked in this part...

Thank you!
Reply
Jon 8 years ago
Hi Kevin,

i've added the following lines to the code:

<span class="media">MEDIA</span>

var img

if (data[i].entities.media) {
img = '<img src="' + data[i].entities.media[0].media_url + '" width="100%"/>';
}

but now i'm getting the error: No data returned, you might want to clear tweets-date.txt

I clear the text file, but the same error keeps coming up..I went through the JS file and nothing else seems to be different.
Reply
Kevin Liew Admin 8 years ago
Hi Jon, what I would recommend is, go to the demo with image, and download the twitter.js from there.
Reply
Andrea 8 years ago
Hello Kevin,

Thank you for the wonderful tutorial series :)

I would like to use the feed generated by a hashtag query as well, but the script with the demo with images you provided won't work... It works with username but not if you enable the hashtag.

Can you please give us some correct code?

Andrea
Reply
mike 8 years ago
can you give me a little more info in how to get the images to import in? Do you have a downloadablt version of that package? Also I cant seem to get get the search version working properly even after deleting the cache data.
Thanks
Reply
Kevin Liew Admin 8 years ago
Please refer to the comment above.
Reply
mike 8 years ago
Well I got the images to work properly but the #nascar wont work, and advice would be greatly appreciated.
Thanks
Reply
Kevin Liew Admin 8 years ago
Hi Mike, I can't test it, but I can guide you to find the images.

Go to line 43 in the javascript, success callback from AJAX, add console.log(data).
(You should use firefox or chrome)...

Then go to the console tab of developer tool, you should able to see it split out the JSON data from twitter. From there, if you know one of the tweet that should contain image, look for it in the JSON, it should be there under media. After that, you just need to get the path of the media and edit the javascript just like what I did in the "demo with images" demo.
Reply
Jon 8 years ago
Hi Kevin,

Are there any changes that need to be made to the tweets-grab.php file to get images to show up in the feed? I've added the img/media code in the JS file, but i keep getting the "clear tweets-date.txt" error.
Reply
Kevin Liew Admin 8 years ago
Just go to tweets-date.txt and remove the content inside. It should work. Are you using hashtag or user account?
Reply
Jon 8 years ago
All good. I figured it out. I'm doing hashtag search, no username. I've also made modifications to show the profile pic as well.

Had to modified the following code to get the images to show:

TWEETS-GRAB.PHP:

$data = file_get_contents($_GET['api'] . '?q=' . urlencode($_GET['q']));

TO

$data = file_get_contents($_GET['api'] . '?q=' . urlencode($_GET['q']) . '&include_entities=' . $_GET['include_entities'] . '&result_type=' . $_GET['result_type']);

TWITTER.JS

if(JQTWEET.hash){request={q:JQTWEET.hash,expiry:JQTWEET.cacheExpiry,api:'http://search.twitter.com/search.json'}

TO

if(JQTWEET.hash){request={q:JQTWEET.hash,include_rts:true,count:JQTWEET.numTweets,include_entities:true,result_type:'mixed',expiry:JQTWEET.cacheExpiry,api:'http://search.twitter.com/search.json'}

var text,name,img,html='<div class="tweet"><span class="text">TWEET_TEXT</span><br /><span class="media">MEDIA</span><span class="time">AGO</span> by <span class="user">USER</span></div>';

TO

var text,name,img,html='<div class="tweet"><img style="float:left" src="PROFILE_PIC" /><span class="text">TWEET_TEXT</span><br /><span class="media">MEDIA</span><span class="time">AGO</span> by <span class="user">USER</span></div>';

if(data[i].entities.media){img='<img src="'+ data[i].entities.media[0].media_url+'" width="100%"/>';}

TO

if(data[i].entities.media){img='<img src="'+ data[i].entities.media[0].media_url+'" width="100%"/>';}else{img='';}

$(JQTWEET.appendTo).append(html.replace('TWEET_TEXT',JQTWEET.ify.clean(data[i].text)).replace(/USER/g,name).replace(/MEDIA/g,img).replace('AGO',JQTWEET.timeAgo(data[i].created_at)));

TO

$(JQTWEET.appendTo).append(html.replace('TWEET_TEXT',JQTWEET.ify.clean(data[i].text)).replace('PROFILE_PIC',data[i].profile_image_url).replace(/USER/g,name).replace(/MEDIA/g,img).replace('AGO',JQTWEET.timeAgo(data[i].created_at)));

everything looks great now!

Thanks for the script!
Reply
Niklas 8 years ago
This is an awesome script! Thanks for sharing, you da man! :D
Reply
Eric 8 years ago
Hi there,

Is it possible to have both options within the menu? So, hashtags and a username?
Reply
team_critter 8 years ago
This is great stuff. Question before I dig in deeper...is it possible to keep 'appending' the latest tweets to tweets-cache.txt instead of overwriting them each time? This would be a great way to track hashtags over time (for visualizations and/or analytics). Thanks Kevin.
Reply
Kevin Liew Admin 8 years ago
Hi, I'm not sure how is it going to affect the javascript, but this is what you can do:

Change this line 40: $datefile = fopen($date, 'a');

It's based on this link: http://www.tizag.com/phpT/fileappend.php
Reply