I am hosting sproutpics on my dreamhost account, which is mostly being unused. I have lots of extra disk space to play with, so I have written a LiveJournal aggregator. The basic concept is similar to SproutSearch (read an XML feed of blog information, store it, organize and display it). Since Dreamhost now offers PHP 5 hosting, I opted to switch sproutpics to PHP 5 so I could use some new functionality it offers.

Instead of using my own HTML parser, as I did with SproutSearch, I used SimpleXML to parse LiveJournal's RSS feed. I found this to work very nicely, it uses a lot less code than SproutSearch, and it's no doubt quite a bit faster as well.

You can see my modest page here:
http://www.sproutpics.com/livejournal.php

When I first created my blog search engine, it was a page within the SproutWorks website. I eventually bought sproutsearch.com and mirrored the content there, while also leaving it on SproutWorks. The traffic to SproutSearch was very low until around May, when I made the search engine organize the blogs into topic pages. The traffic to both sites climbed, with SproutWorks getting a lot of traffic (for me, anyway). At some point, SproutWorks traffic dropped considerably, while SproutSearch continued to climb. Now, they get about even traffic levels.

I am assuming that one, or both of these sites have been penalized because they share the same content. So, today I am only listing a few blogs in each topic on SproutWorks, while keeping the full content on SproutSearch. I am hoping that SproutSearch will gain at least as much traffic as I will lose on SproutWorks.

I thought of a neat idea the other day, and I made it a reality. I wrote a PHP script that reads Yahoo's Buzz Index site, where they list popular search terms. The script adds all the search terms to my LiveJournal topics database. Then it searches all the indexed LiveJournal posts for those topics, and saves them in the index. The result is hundreds of new automatically generated pages that are related to popular things on the Internet.

Check out the results here:
http://www.sproutpics.com/livejournal.php

I have created a new program that tracks price flucuations in the BlogShares fantasy market. The basic functions are working, and more sophisticated functions are planned. The graphs are created in real time by PHP.

http://www.sproutworks.com/blogsharesideas.php

I made some improvements to the graphing program, it now displays the change percentage.

A while back, I added tagging to my forum/blog program. I included tags in my RSS feed so that Technorati and other sites could categorize my posts. Today I've made a Tag Cloud page, which makes pages of articles with the same tags. These pages should make my contextual advertising work better.

I just added an AJAX blog browser to SproutSearch. All the functionality isn't there yet, but its a neat way to browse blogs.

http://www.sproutsearch.com/blogbrowser.php

I have started working on an RSS search engine to add to SproutSearch (http://www.sproutsearch.com). I used SimpleXml to parse the RSS. This makes it really easy to parse them. The general idea I have is to store items from the feeds in a database, along with their tag information. Then I can generate a bunch of tag pages once I have gathered enough items.

The tags will have their own pages on SproutSearch, but I am also thinking of combining all the different sources into one page. So, on a topic page, it might have some listings from Blogger, some from LiveJournal, and some from RSS. As I add more data sources, those can be integrated as well. Then it would have links to view a more detailed listing of a particular data source.

My blog and RSS feed search engine is almost to the point of indexing 5 million blogs from Blogger.

I have implemented a new rating system. When you click on a blog, please vote on it.

http://www.sproutsearch.com

I got an email from Feedblitz saying that they are adding some new features to the free accounts. I have customized the appearance of the email with some graphics I had laying around. I'll try to make a more extensive email template later.

I just noticed that the Blogshares Price Tracker has not been updating for a couple weeks. When I looked into it, I discovered that something happened on the server so that the include files were not being found. A Dreamhost tech pointed me to their wiki page about cron jobs, where I found I had to change the path to PHP.

After I got the updates going again, I rewrote a few key parts that were making the price tracker very slow. There were some SQL queries that were inefficient, so I rewrote those, and moved some data into different tables. Now the industry listing and details pages are much faster. I made the graphs a little faster, but they are still way too slow. The problem is that the graph script queries a table with over 10 million rows. I'll try to think of some ways to speed it up.

I have just optimized the performance of the Blogshares Idea Tracker's graph rendering program. The graphs are loading much faster now. The industry listing page now allows you to load the graph for any industry by clicking a button.

My blog search engine SproutSearch is now indexing over 8 million blogs. I am now working on changing the way the blogs are ranked. For now, they are sorted by the sheer amount of content they contain. I noticed a big problem with this method is that many spam blogs contain masses of content. I don't like SproutSearch linking to so much spam, so I need to find a way to remove a lot of these listings.

It is not practical for me to read 8 million blogs, so I need to come up with an automated method to detect spam. Many spam blogs use the same words over and over. So I wrote a program to count the number of repeated words. Most spam blogs seem to use a similar number of words per post. I made another program that computes the standard deviation of the number of words in a post. Using these metrics, I will make a program that flags potential spam so I can review and delete it.

www.problogger.net/archives/category/birthday-prize-giveaway/

ProBlogger is having a birthday bash with $54,000 in prizes. ProBlogger is a blog dedicated to the topic of making money with blogging. It is a tantalizing idea, but I tend to spend more time programming my site than writing on it. Still, I enjoy reading the tips about blogging, which I may actually implement someday.

I am writing this post as my entry to win 2 LG 20" USB LCD monitors provided by DisplayLink (www.displaylink.com) That would be an awesome compliment to my new 22" Viewsonic LCD.

So, everyone check out ProBlogger, it's an interesting read.