I have been hearing about Ruby on Rails for last few months, so I finally thought I'd check it out. Ruby is a programming language, and rails is an application framework. For about the last 2.5 years, I have been writing my web applications in PHP. It's been working well for me. I feel like I'm relvatively proficient at it.

I started writing a content management system as I was learning PHP. As a result, it contains mixed programming styles. In the early days, I wrote procedural programs, and put all my scripts in my web root directory. Later I got a little smarter and made a directory structure of my scripts. I also started using classes and objects. This made things a lot easier to re-use later.

At some point I started using a template engine I got from a tutorial, and modified it to my liking.

My CMS now is around 25,000 lines of code, and only the newer parts are organized very well. I am always thinking of how I could improve the way it is all organized. After completing my first rails tutorial, I got some inspiration for how to overhaul my CMS. I need to build a moderately complex app with rails to get a clearer picture of what I need to do.

So, I think I will write an application framework for PHP, which borrows some concepts from rails. I like the way rails configures your .htaccess file to make simple urls. It can also make sense of database tables without needed to code any database specific stuff.

I am hosting sproutpics on my dreamhost account, which is mostly being unused. I have lots of extra disk space to play with, so I have written a LiveJournal aggregator. The basic concept is similar to SproutSearch (read an XML feed of blog information, store it, organize and display it). Since Dreamhost now offers PHP 5 hosting, I opted to switch sproutpics to PHP 5 so I could use some new functionality it offers.

Instead of using my own HTML parser, as I did with SproutSearch, I used SimpleXML to parse LiveJournal's RSS feed. I found this to work very nicely, it uses a lot less code than SproutSearch, and it's no doubt quite a bit faster as well.

You can see my modest page here:
http://www.sproutpics.com/livejournal.php

I read an article on slashdot today about a company that analyzes blogs for trends related to corporate products. In the article, US Cellular found out that there is a lot of teenage angst related to running out of minutes on their cell phones. US Cellular decided to give people more minutes based on their blog research.

It's interesting to see the blogosphere gaining influence on corporations and the media. When computers get enough grammical sence and analytical ability to interpret blogs, we'll be able to gauge public opinion on just about everything, without the need to conduct surveys.

I am anxious to do some data mining of my own, my blog database has over 1.7 million entries so far. A relatively easy thing to do would be finding the most frequently used words among different topics. I think I'll start with that, and think of other interestic metrics to calculate.

I heard about Google Sitemaps shortly after it was released, and I decided to give it a try. I wrote a simple PHP script that maps a sitemap containing all my blog topics pages. Next, I added the sitemap file in my Google Sitemaps account. Since then, Google has requested my sitemap file a few times. I have not looked at my server logs to see what GoogleBot is indexing. Strangely, traffic to this site is way down from its peak earlier this month, and sproutsearch.com traffic is up. I figure maybe it's because of the similar content.

I also tried using a unicode symbol character of a sun in my page titles, but I have since removed it. I left the symbol on sproutsearch, however. I am waiting to see if it shows up in search results when Google updates its cache of sproutsearch.

Today I converted the SproutSearch database from InnoDB to MYISAM. I wanted to use fulltext searching, which InnoDB does not support. I didn't really need transactions either. The indexes now use about 100M less space.

I rewrote my search queries to use the new fulltext index. My phrase building script now runs a lot faster, and I have been up all night adding keywords for it to find.

So what I have so far is a little over 900 pages of keyword pages which display related blogs. The next step is to create a directory structure to organize the keywords into similar groups. Right now these pages are all on a list, which is more for search engines to look at than people. I am hoping the search engines will gobble up the new content.

I have recently made some advances on SproutSearch, my blog directory site. SproutSearch has helped me get a bit more traffic, because it adds massive amounts of content to my site. At the moment, it contains about 1.3 million links to blogs.

When I set out to write SproutSearch, it was an exercise in site scraping. Using an HTML parser I had developed earlier, I made a simple program that parsed Blogger's recently updated blogs xml feed. It stored all the blogs it could find in a database. I set up a cron job to run this script every ten minutes. Even when I am sleeping, my website is finding new blogs for me.

Next I made a program to sort the blogs alphabetically, and put them in organized pages. As the number of blogs grew, so did the amount of time my script needed to display a page. I read an article in PHP Magazine about caching, and integrated a caching script into my site. Now, each page would be saved in a file so that it could be recalled quickly.

I left things this way for a while, and it was working well. I started to get more traffic, which was one goal of this project. I noticed that the first page of each letter was geting most of the inbound traffic. For some reason, the search engines won't point people to the hundreds of additional pages that are not #1.

I might try to make some pages with a lot more blogs per page, maybe about 10,000. Then I will see if these big pages generate more traffic than my 500 blog pages.

The other day, I wrote a program to perform searches on my directory and turn them into web pages. I typed in a whole bunch of popular keywords and let it generate pages. I am hoping that these keyword pages will attract a new wave of traffic of people searching for niche terms.

The next step for me is to categorize the topic pages I have been generating.

There has been a lot of buzz about this "AJAX" technology among web designer types. AJAX is a relatively new method of live communication between a web browser and a web server. AJAX is implemented with a client side scripting language, and a server side scripting language.

In my implementations of AJAX, I use JavaScript and PHP. So far I've only created a few trivial examples, but I hope to soon create something a bit more substantial.

My current AJAX projects are:

The chat room on this site. Messages are sent and retrieved as needed, avoiding wasteful page refreshes.

The counter that displays how long you have been viewing this site is another simple AJAX trick. It is a JavaScript that reports timings to the server, which tracks timing values across several pages.

If you click on this site a few times, you may notice that there are lines being drawn in the background. A script is transmitting your clicks to the server, and when you load a new page, all of your clicks are drawn into the background image using PHP and GD. It is not meant as a serious tool, at least not until I have added an inferface to control the drawing methods.

I have only created a few trivial examples of what AJAX can do, but I see great things on the horizon, for my own website and the web in general.

Check out http://maps.google.com for a really cool example of how intelligent browser/server communication can make a common task more intuitive.

I wrote a simple script to display a random blog that changes every 10 seconds. It worked on my computer but wouldn't work on this site. I spent a lot of time trying to get it to work, and I just found out that using order by rand() in a MySQL query can be such a resouce hog that it doesn't work at all.

I don't know how the internals of MySQL work, but I think it creates a temporary table, and then sorts it randomly, which is a huge waste. When you have over a million rows, and all you want is one randomly chosen row, MySQL is doing millions of times more work than it needs to.

My workaround was to choose a random number, and then use a limit clause to start at that number, and only return 1 row.

I've been having an intermittant problem with PHP sessions. Sometimes when posting a message it will log me out first. I made a test script that has the message post form in it and kept posting stuff, but it never logs out in that case. I changed my login script a little, hopefully that will fix it.

I haven't really been into creating 3D graphics since high school. In high school I used Strata Studio Pro on the Mac to create 3D stuff. Tonight I downloaded the free Maya Personal Learning Edition 6. I completed the first 2 tutorials. In the first one I designed a simple temple. In the second tutorial I designed a hammer.

When I first opened up Maya, I was frustrated looking at all the menus and buttons. Plus, the Maya mentor would not work for me. Now I know how to edit some basic stuff.