Some may or may not know already, but I was asked about the CommunitySEO development team to install the software on a site of mine, and to review it overall. I've been given access to the source code, under an NDA, and will be reporting back on that a bit later after I can look through it more thoroughly. For now though, I wanted to go ahead and start the review of my experience so far.

I'm attaching my relevant awstats statistics I pulled for the last few months, as well as my site: search results in Google, MSN, Yahoo, and Ask.com. Here are the highlights though

Google:
site:bfarber.com 38,900
link:bfarber.com 211

Yahoo:
site:bfarber.com 10
link:bfarber.com Inlinks 13,092

MSN:
site:bfarber.com 1,183
link:bfarber.com 0
linkdomain:bfarber.com 0

Ask.com
bfarber.com 683


The site has been up for a few years now (and note, I no longer own it, however the owner is cool and has let me install this module anyways). I noticed that it doesn't play too nicely with my custom portal due to edits to index.php. I'll have to have a look at that soon, as my forum index isn't immediately accessible right now. The site has 61,846 members and 38,359 posts. The module was installed June 5th give or take.

Month	   Unique	Total	 Page Views  Hits	  Bandwidth
Jan 2007	85271	99384	521372	6720562	492.37 GB
Feb 2007	111484	131230	949934	6820053	412.73 GB
Mar 2007	52902	62758	597369	2302845	68.20 GB
Apr 2007	45069	54094	617394	2570017	97.88 GB
May 2007	72576	86803	699900	3340855	102.91 GB

And spider hits for May

May Robots/Spiders visitors
18 different robots	Hits	Bandwidth	Last visit
Googlebot	73357+24	1.35 GB	31 May 2007 - 23:54
MSNBot	35025+1507	1.80 GB	31 May 2007 - 23:57
Unknown robot (identified by 'spider')	18856+1080	1.05 GB	31 May 2007 - 22:05
Unknown robot (identified by 'robot')	8570+58	123.88 MB	31 May 2007 - 23:55
Voila	3445+4727	46.85 MB	31 May 2007 - 23:51
Unknown robot (identified by 'crawl')	7531+136	361.21 MB	31 May 2007 - 23:57
Inktomi Slurp	682+576	9.83 MB	31 May 2007 - 21:41
Unknown robot (identified by hit on 'robots.txt')	0+1081	324.84 KB	31 May 2007 - 23:39
Internet Shinchakubin	960+2	30.57 MB	31 May 2007 - 17:24
EchO!	960	17.26 MB	31 May 2007 - 20:19
Scooter	146+2	1.53 MB	15 May 2007 - 04:13
AskJeeves	69+35	3.97 MB	29 May 2007 - 06:06
Alexa (IA Archiver)	0+47	14.14 KB	31 May 2007 - 00:39
Lycos	8+5	428.46 KB	27 May 2007 - 18:01
Voyager	6+4	256.30 KB	30 May 2007 - 01:07
Walhello appie	4+4	376.88 KB	18 May 2007 - 19:17
Netcraft	3	282.42 KB	25 May 2007 - 06:29
Nomad	1	163.29 KB	04 May 2007 - 05:30

there are more details in the attached file (Jan - May plus inbound links from search engines data for Jan-May).


Now, it's only been a few days, not even a week, and already Google seems to be mostly updated. I'm really quite surprised at how quickly this has happened. The entire first few pages of results are completely .html rewritten pages. I had to get to about page 12 to see dynamic links again. I'm rather impressed at how quickly Google got the new links added. I am not using webmaster tools on bfarber.com presently, but may check it out down the road to see what it can offer.

The number of pages in Google has gone down it seems, to 38,500, however that could very well be due to removing duplicate content too. I'm going to keep my eye on it and see what happens. Yahoo still has old data. Not sure if I need to submit my site to Yahoo or something? MSN still shows old links still too.

I will continue posting updates as time goes on and there's more to really see. I'll also post my review of the code here soon too. I need more time to go through it completely though.

I also managed to fix the bug I noticed where I couldn't access the forum index. My custom portal was loading up instead. It did not require any edits to the community seo files - just a couple to index.php.

It's nice for you to do this Brandon, as one of IPB's primary coders. Very reassuring

I'm using your portal as a frontpage, if I install this product will I get in trouble?

Now that I see that Bfarber is here messing with this product that is quite a bit of re-assurance in the product. I have just set aside to budget for this software for my site at the end of next month....so here is another sale based off of bfarber!

Thanks
William
SoCalSportRiders.com

Yes, I can say this as well. Seeing bfarber here testing the product made my decision. And I probably will buy another licence soon.

Ok, as promised here is my monthly update. A note: we did *nothing* besides install community seo during this phase. I DID also fix an issue regarding my portal, which I detailed in one of these forums, and I also added 301 redirects for some of the older style URLs to access links in the download manager (if anyone needs those details I can supply them). We updated PHP on the server, but no other software was installed. I did not submit my sitemap to any search engines either. I intend to do that soon.

Links in MSN have gone up, while in other search engines I'm seeing less. I don't see this as a problem, however, because it would seem that it's duplicate content being removed.

Google:
site:bfarber.com 30,500
link:bfarber.com 200

Yahoo:
site:bfarber.com 11
link:bfarber.com Inlinks 15,993

MSN:
site:bfarber.com 1,354
link:bfarber.com 0
linkdomain:bfarber.com 0

Ask.com
bfarber.com 344


We did have a problem for a couple days that may have played a part as well. PHP was updated and ioncube wasn't reinstalled - so we got an error about the ioncube loaders for a day or two until the host fixed that.

Most of the links in Google are now the static versions. I note that some have the mode= parameter still. I would love to see an option to redirect those to the default mode on the site. Very few people actually use mode=threaded in all reality, and I would be ok with shutting down the option entirely with the result being that none of those get indexed (or rather, they get redirected to the default viewing mode).

Likewise I would like to see an option to put the filename for downloads in the page name. I submitted that here: http://communityseo....inks-t1398.html

Traffic was slower this month than normal, however it is comparable to June 2006. Traditionally, traffic spikes a little again this month (July), goes down in August, and jumps up again in September and October. I will try to post reports after each month to see how this module affects things.

While some search terms bfarber.com has not changed in, the two top search terms we have gone up significantly.

If you recall, here was our search term stats prior to the module being installed

Quote

Here is our current ranking

Quote

Whereas we were not even listed in the first 30 results for "login screens" before, we are now number 9, putting us on page 1 in google. Additionally, we have gone up to number 1 for "xp login screens". This is outstanding. The other search terms, while we do target for them partially, have not traditionally been as well-ranked for our site anyways. I am hoping to see improvements on them over the next few months however.

62,113 members
38,397 posts

We do not have a lot of posting (most people register to download). Registration has gone up. 288 registrations in June, compared to 257 in May and 187 in April. In reality, posting stats have gone up drastically for June too. 37 posts in June compared to 12 in May and 22 in April. These stats taken from the IPB ACP Statistics Center. Same situation with new topic stats - 14 in June compared to 5 in May and 10 in April.

Overall, activity is apparently up, even if traffic is done. This would tend to make me believe that we have more quality traffic now.

June
42746 unique visitors
50612 total visits
553807 page views + 302014 not viewed
2296671 hits + 350004 not viewed
92.31 GB bandwidth

Month	   Unique	Total	 Page Views  Hits	  Bandwidth
Jun 2007	42746	50612	553807	2296671	92.31 GB

Jun Robots/Spiders visitors
18 different robots	Hits	Bandwidth	Last visit
Googlebot	75505+51	1.28 GB	30 Jun 2007 - 23:58
MSNBot	16177+1447	932.02 MB	30 Jun 2007 - 23:56
Unknown robot (identified by 'robot')	9847+88	288.17 MB	30 Jun 2007 - 23:59
Unknown robot (identified by 'crawl')	8301+193	144.17 MB	30 Jun 2007 - 21:06
Voila	1431+2017	15.92 MB	30 Jun 2007 - 23:59
Unknown robot (identified by 'spider')	2926+400	145.25 MB	30 Jun 2007 - 21:06
Yahoo Slurp	884+858	10.95 MB	30 Jun 2007 - 23:59
Scooter	1120+14	10.35 MB	27 Jun 2007 - 20:05
Unknown robot (identified by hit on 'robots.txt')	0+852	248.83 KB	30 Jun 2007 - 23:59
Internet Shinchakubin	789+2	49.89 MB	30 Jun 2007 - 18:22
echo	435+1	15.95 MB	30 Jun 2007 - 23:55
Alexa (IA Archiver)	89+64	7.34 MB	30 Jun 2007 - 08:55
Ask	105+45	5.69 MB	30 Jun 2007 - 23:53
Lycos	10+7	469.83 KB	13 Jun 2007 - 11:42
Voyager	6+9	315.18 KB	30 Jun 2007 - 00:13
Walhello appie	2+3	20.42 KB	26 Jun 2007 - 22:34
LinkWalker	0+1	298 Bytes	18 Jun 2007 - 12:39
Nomad	1	1.03 MB	26 Jun 2007 - 01:18


Jun Links from an Internet Search Engine
30 different refering search engines	Pages	Percent	Hits	Percent
Google (Images)	35302	78 %	82975	81.3 %
Yahoo!	6784	14.9 %	14516	14.2 %
Google	1698	3.7 %	1799	1.7 %
AltaVista	459	1 %	463	0.4 %
MSN Search	277	0.6 %	553	0.5 %
Unknown search engines	261	0.5 %	491	0.4 %
AOL	246	0.5 %	387	0.3 %
Dogpile	56	0.1 %	100	0 %
Ask	54	0.1 %	86	0 %
Netscape	22	0 %	36	0 %
Excite	21	0 %	195	0.1 %
AllTheWeb	20	0 %	117	0.1 %
MetaCrawler (Metamoteur)	9	0 %	9	0 %
WebCrawler	8	0 %	8	0 %
DMOZ	4	0 %	4	0 %
AOL (fr)	4	0 %	8	0 %
Search.com	3	0 %	4	0 %
Earth Link	2	0 %	2	0 %
search.ch	2	0 %	2	0 %
InfoSpace	2	0 %	2	0 %
Go.com	2	0 %	125	0.1 %
ix quick	2	0 %	9	0 %
Yandex	2	0 %	2	0 %
Aport	1	0 %	1	0 %
Tiscali	1	0 %	23	0 %
Club-internet	1	0 %	1	0 %
Baidu	1	0 %	31	0 %
Lycos	1	0 %	46	0 %
Rambler	1	0 %	1	0 %
Mamma	1	0 %	1	0 %


Jun Search Keywords (Top 25)
evil	554	2.9 %
clown	553	2.9 %
windows	538	2.8 %
wallpapers	448	2.3 %
tia	404	2.1 %
wallpaper	398	2 %
playboy	387	2 %
xp	365	1.9 %
bunny	345	1.8 %
carrera	321	1.6 %
vida	320	1.6 %
akuma	259	1.3 %
boondock	258	1.3 %
saints	258	1.3 %
guerra	250	1.3 %
blue	226	1.1 %
kittie	219	1.1 %
killswitch	207	1 %
theater	200	1 %
dream	198	1 %
star	193	1 %
engage	192	1 %
wars	180	0.9 %
electra	170	0.8 %
carmen	169	0.8 %
Other words	11451	60 %

Some things to point out from these statistics compared to May:

• Activity is up for Google. Surprisingly, Google is using less bandwidth however - this must be a result of the new Spider skin. Hits and bandwidth for May: 73357+24 1.35 GB. Hits and bandwidth for June: 75505+51 1.28 GB
  
• Hits are up and down for various spiders. Some have increased activity while some have decreased activity. Mostly, activity has gone up.
  
• Incoming links from Google have increased slightly, despite the lower activity/traffic level. For most other search engines, incoming links have decreased, which coincides with the slower traffic.
  
• Most search terms stayed relatively stable/the same.

I also thought this statistic was interesting - look at the number of redirects that have been sent this month.

301	Moved permanently (redirect)	149320	65.9 %	556.15 KB
302	Moved temporarily (redirect)	26362	11.6 %	5.68 MB

All in all, there hasn't been much change in June yet, however Google has nearly updated all their links to the site, and our ranking for our two top key phrases have shot up drastically. I expect even better results in July, especially as I sign up for webmaster tools and see what Google can help with.

See next post for details of my review of the code

Now, I signed an NDA so I can't get into many of the details, but I agreed to write up a review of the code as I was provided an unencoded copy. The developers felt that if I took a look at it and gave my ok, it might ease some people's concerns about using an encoded product.

File Edits: The file edits that are needed are unfortunately unavoidable. It helps me seeing this where we can add useful hooks down the road so developers can hook into the software without modifying it, but even then it is unlikely we'd be able to capture all locations a hook might be useful.

There are many output exit points in IPB, and because this module operates on the output, each needs to be hooked into. This is the reason for edits to class_display.php and class_ajax.php for example.

All in all, the edits are likely as streamlined as they get for now.

Code Execution: The coding is great - it is object-oriented, just like IPB itself, and it thoroughly uses IPB's objects and methods. For example, it does not create a new database connection to grab data like similar topic - instead it uses the existing connection (as it should). It is very organized and clear (though this should hardly be a factor in anyone's purchase of the software, given that they can't see the code anyways).

The rewritting methodology is really what is impressive. I expected to see 100 preg_replace statements in the main rewritting function, but that is hardly the case. They use a couple to get all the topic ids and member ids into an array so that the titles and member names can be loaded from the database. Then one preg_replace is run on the content to replace the URL's. They use the preg_replace_callback function, which is considered "better" than preg_replace with the /e modifier for security reasons, which is good. The callback function is very genius in how it determines how to rewrite urls. I must say, they have implemented the absolute best rewriting method I've seen to date (I have looked at FURL by kim, and Minerva Alpha releases). This is likely why there is no overhead in loading the pages (at least in the current release - I hear past releases were not so fortunate).

All in all, I give the coding 2 thumbs up, and kudos for the creativity.


Areas of Improvement: What kind of review would this be without some constructive criticism?

More "titles" in the URLs - I'd like to see the filename for downloads added to the link, and I'm assuming we'd see the same thing for blog entries and gallery images too.

Plugin support - it would be nice to be able to use this addon to rewrite other areas of the site, and/or for developers to hook into somehow. For example, it would be trivial for me to update my links system to use this module, if it did not require me to edit the already encoded file.

Documentation - would be nice to see a comprehensive documentation of what each and every setting in the ACP does, and which settings you do not want to set to conflict with other settings. SEO is like voodoo for most people, so the more information that CommunitySEO can provide, the better.

Redirects - this is a tough one. I've seen some of the bugs reported and reviewed the code that does redirects as a result. Essentially what I think would need to happen is to have a setting that says "only allow one url per page" and then any other incoming links to that page not using the one url would be redirected to the correct url. This would mean that no mode= links would work. Also, there is that problem with the title anchoring (jumping to posts) which is tricky because IPB uses p= sometimes and gopid= sometimes and pid= sometimes, and so on. All in all, it seems like a decision just needs to be made as to which functionality can be sacrificed. Could make it all configurable too.

Sitemap - while I see how and why it is made the way it is, I'm not fond of it being recycled completely at intervals. I would suggest looking into ways to improve this. This isn't a huge deal really, but would be my recommendation.

Feature Requests: Might be nice to see plugin and hook support, as well as possibly an HTML sitemap. HTML sitemap would be essentially the lofi version though, so not sure if it's worth it.

I would like to see configurable robots.txt from the ACP, and maybe configurable rewriting options? That would be tricky though I imagine.


Final Opinion: This really is an excellent product. I would advise anyone worried about using an encoded product not to be. There are no callbacks in the software (except during the installation if you check the "Validate my key" checkbox) and the code-style is on par with any other top ipb modder or php developer. I saw no security issues in reviewing the code either.

SEO is one of those things that's hard to pin down too, but I think in terms of "what can a software package add to my site to help my search engine ranking/positioning" this product has just about everything that you can think of or need. All I would like to see now is more polishing and attention to minor details. I can't wait for the next major version.

Nice review bfarber, I can agree about this one also:

"Plugin support - it would be nice to be able to use this addon to rewrite other areas of the site, and/or for developers to hook into somehow. For example, it would be trivial for me to update my links system to use this module, if it did not require me to edit the already encoded file."

Thanks again for taking the time to do this!

That was a full nights worth of work, eh?


Thanks that was a great review and it really does confirm, my choice to purchase later this month!

Thanks Bfarber.

What is NDA?

BobSuch, on Jul 5 2007, 10:45 PM, said:

Non Disclosure Agreement.

In other words, he can't leak any of their secrets or pass around the unencoded version.

How this month looking so far?

Time for an update me thinks.

Google:
site:bfarber.com 19,400
link:bfarber.com 197

Yahoo:
site:bfarber.com Seems to be broken?
link:bfarber.com Inlinks Seems to be broken?

I can't seem to do site: and link: queries through Yahoo anymore - just gives me a blank white screen

MSN:
site:bfarber.com 4542
link:bfarber.com 0
linkdomain:bfarber.com 0

Ask.com
bfarber.com 328


It seems like MSN has really picked up their indexing. They've jumped over the past month. Also, Google slowly trims out all the duplicate content - I don't hardly see any duplicate content for my site in Google anymore, outside of my portal which can load via an autocom= link, and the home.html built in link. I could fix that I suppose if I had the ambition.

Search term position presently

Quote

This is about the same as last month. Through Google's Webmaster Tools I found other search queries I was ranking for, as outlined above, that were similar to what I was searching for previously. After my experiments here are done, I should probably find a way to edge in on the actual terms I originally searched for, as that is what a normal user would probably search for.

July
23865 unique visitors
28598 total visits
485521 page views + 265901 not viewed
1837000 hits + 310666 not viewed
79.00 GB GB bandwidth

Month	   Unique	Total	 Page Views  Hits	  Bandwidth
Jul 2007	23865	28598	485521	1837000	79.00 GB

Traffic has obviously gone down a little in the last month. I'm not sure what to make of that just yet, though if I had to guess - it's likely due to more people adopting Windows Vista and having less interest in customizing XP. The site will likely need to take new direction very soon to stay alive.

Robots/Spiders visitors 	 
21 different robots	Hits	Bandwidth	Last visit
Googlebot	55824+39	1.74 GB	31 Jul 2007 - 21:51
MSNBot	23649+1349	1.18 GB	31 Jul 2007 - 23:59
Unknown robot (identified by 'robot')	10385+88	326.21 MB	31 Jul 2007 - 20:59
Unknown robot (identified by 'bot/' or 'bot-')	8230+150	368.09 MB	31 Jul 2007 - 17:56
Unknown robot (identified by 'crawl')	7042+295	80.30 MB	31 Jul 2007 - 23:11
Voila	1019+1417	10.54 MB	31 Jul 2007 - 23:34
Unknown robot (identified by 'spider')	1578+521	76.54 MB	31 Jul 2007 - 23:58
MSNBot-media	1544+134	89.92 MB	31 Jul 2007 - 23:19
Yahoo Slurp	718+267	8.66 MB	31 Jul 2007 - 23:40
Scooter	781+12	6.14 MB	30 Jul 2007 - 09:38
Yahoo! Slurp China	276+321	2.77 MB	31 Jul 2007 - 20:35
Unknown robot (identified by hit on 'robots.txt')	0+587	170.83 KB	31 Jul 2007 - 22:44
Internet Shinchakubin	531	35.34 MB	31 Jul 2007 - 22:07
WebCollage	250	7.38 MB	31 Jul 2007 - 23:03
Ask	148+56	11.92 MB	30 Jul 2007 - 15:02
Alexa (IA Archiver)	158+24	7.71 MB	31 Jul 2007 - 23:35
Lycos	33+21	1.81 MB	31 Jul 2007 - 17:33
Voyager	22+17	39.07 KB	30 Jul 2007 - 07:06
echo	4	336.99 KB	01 Jul 2007 - 00:14
The web archive (IA Archiver)	0+3	894 Bytes	23 Jul 2007 - 03:23
ZyBorg	2	164.08 KB	09 Jul 2007 - 08:05


Google (Images)	9148	46 %	31337	58.1 %
Yahoo!	6431	32.3 %	13988	25.9 %
Google	1944	9.7 %	2315	4.2 %
Google (cache)	641	3.2 %	2079	3.8 %
Windows Live	530	2.6 %	1799	3.3 %
MSN Search	511	2.5 %	855	1.5 %
AltaVista	317	1.5 %	317	0.5 %
AOL	94	0.4 %	147	0.2 %
Dogpile	46	0.2 %	47	0 %
Ask	39	0.1 %	70	0.1 %
Unknown search engines	32	0.1 %	51	0 %
AllTheWeb	31	0.1 %	150	0.2 %
ix quick	23	0.1 %	176	0.3 %
Excite	13	0 %	106	0.1 %
GoodSearch	12	0 %	19	0 %
WebCrawler	11	0 %	11	0 %
MyWebSearch	8	0 %	8	0 %
AOL (fr)	5	0 %	10	0 %
MetaCrawler (Metamoteur)	5	0 %	5	0 %
DMOZ	4	0 %	4	0 %
Stumbleupon (Social Bookmark)	4	0 %	50	0 %
del.icio.us (Social Bookmark)	3	0 %	3	0 %
Baidu	3	0 %	39	0 %
Yandex	3	0 %	3	0 %
Earth Link	2	0 %	2	0 %
Search.com	2	0 %	2	0 %
Vivisimo	2	0 %	2	0 %
Netscape	2	0 %	4	0 %
Clusty	2	0 %	2	0 %
My Search	1	0 %	1	0 %
Kartoo	1	0 %	1	0 %
Web.de	1	0 %	1	0 %
Aport	1	0 %	1	0 %
Digg (Social Bookmark)	1	0 %	1	0 %
Alexa	1	0 %	1	0 %
Free.fr	1	0 %	1	0 %
Seznam	1	0 %	1	0 %
InfoSpace	1	0 %	1	0 %
Tiscali	 	 	10	0 %
Lycos	 	 	52	0 %
alice.it	 	 	16	0 %
Sapo	 	 	6	0 %
Spray	 	 	1	0 %
Interia.pl	 	 	1	0 %
Go.com	 	 	146	0.2 %
Sogou	 	 	2	0 %


Top 25 Keywords
1508 different keywords	Search	Percent
windows	665	3.2 %
xp	627	3 %
evil	618	2.9 %
clown	617	2.9 %
playboy	518	2.5 %
bunny	497	2.4 %
wallpapers	470	2.2 %
screens	440	2.1 %
wallpaper	392	1.9 %
login	354	1.7 %
vida	294	1.4 %
tia	287	1.3 %
boondock	229	1.1 %
saints	229	1.1 %
guerra	227	1.1 %
patriotic	223	1 %
blue	213	1 %
akuma	203	0.9 %
bfarber	195	0.9 %
victorias	192	0.9 %
secret	191	0.9 %
killswitch	184	0.8 %
logonxp	181	0.8 %
engage	180	0.8 %
file	173	0.8 %
Other words	12207	59.2 %

Some robot activity is going up, while some is going down, just like last month. I haven't really had time to analyze any of this much, nor have I played much with the Webmaster Tools (though I did sign up!).

Hi Bfarber and others,


Is there any possiblity in the near future to upgrade CommunitySeo and/or "Links System" in order to have fURLs in that mod???

PabloC
Madrid

3 months since the last update, come on bfarber, give us one more

Since I just posted this topic about MSN, I decided to check search.live.com.

site:bfarber.com :: 20,700

Last reported from bfarber above was 4542.