Train or Checkpoint?

Caress made the ultimate statement while playing The Darkness for XBOX 360 the other day.

"They really should put a checkpoint at the end of the tunnel!"
 
-- Caress Redd

I agree...  they really should...

Graffiti Beta 1 - Review (part 2): Performance

I finally ran some simple performance tests on Graffiti Beta 1. By simple I mean that only a stopwatch and web browser were used. The purpose was to get a general "feel" for Graffiti's performance with a large numbers posts in the database.

I ran the test on a Dell Inspiron Laptop (2GHz Core Duo, 2GB Ram, 7200 RPM HDD) running Windows Vista 32bit, IIS 7, and SQL Express 2005.

Since I was using my own development laptop, and it has limited free HDD space, I changed the DB recovery mode to "Simple". No other database tuning was performed.

To generate the data I just created a SQL Schema project in Visual Studio 2008 Team System then created a simple data generation plan. The generated data included 5 categories, 100,000 posts, and 200,000 comments. The posts were split evenly among the 5 categories. Comments were also split evenly among the posts (2 comments to a post). 

Results:

100k posts and 200k comments:

• Graffiti was able to display the user facing pages with only a minor performance impact.
   
• The initial load of the site's home page took nearly 5 seconds.
    
• Refresh of the home page took less than 1 second.
   
• The initial load of a category page took about 2 seconds, with refreshes being nearly instant.
   
• Clicking the "older posts" button in any content list took about 4 seconds. Once loaded though, revisiting those pages was nearly instant.
   
• Loading a content item's view took about 1 second on first load, and was nearly instant thereafter.
   
• Loading the control panel resulted in a SQL Timeout error every time.
  

  Looking at the queries in SQL Profiler: It performs a select count against the posts table, probably for use by the summary chart on the dashboard.
    
  Testing the query manually: about 1 minute and 30 seconds to count through the table using the same where and group by clauses. Counting the posts without any where or group clauses still takes around 30 seconds. HDD throughput was at 100% during the entire duration of the query's execution.
   
  This is an area where a full version of SQL server on a dedicated piece of server hardware would make a huge difference. This kind of query bottlenecks on the low throughput of the laptop's hard drive... Low HDD performance is a typical problem on any laptop though.

 50k posts and 100k comments:

• User facing pages loaded with no noticeable performance problems. Speeds were a little snappier than with 100k posts especially on initial page loads and when using the pager in content lists.
   
• The admin dashboard page loaded in 17 seconds on the first load, and was nearly instant on subsequent loads. The "posts" tab of the admin section was the slowest, taking about 28 seconds on the initial load, and about 9 seconds on subsequent loads. The "comments" tab of the admin tools was surprisingly snappy, taking only 3 seconds to load initially.  

Conclusion:

Graffiti beta 1, without special optimization, should be able to serve the needs of most realistic sites in it's target market.

Even on a modestly equipped laptop, performance did not significantly degrade until the number of posts exceeds the 50k mark. On real server hardware it would probably be able to handle well over 100,000 posts with little problem for the end users. The admin tools will degrade more significantly than user-facing pages.

Oddly, the number of comments seemed to have little impact on overall performance.

Looking at the queries that had the most significant performance problems, it will be rather trivial for Telligent to either optimize the queries for larger numbers of posts, or provide alternate configurations that trade convenience features (such as daily report generation rather than real-time reporting) for better performance.

Reality Checks:

• If you actually have anywhere near 100k posts, you probably are not going to be using any off-the-shelf product without significant customization and DB tuning anyway. Keep in mind that it takes 273 posts a day to get to 100k in a single year. Only a large news outlet is likely to generate that quantity of content and most of them would still take a few years to break the 100k mark.
   
  However, 100k comments isn't too terribly unrealistic for a very popular site.
   
• There is one major difference between this test and a production environment of Graffiti. In a production situation, Graffiti would have created one folder on the file system for each post that was created via the GUI tools.  Within each generated folder would be a generated "default.aspx" file. The main purpose of these files is to give IIS a default document to use since the graffiti URLs don't specify actual page names. Graffiti also stores a little bit of metadata in these files that can help it render the posts faster (it doesn't seem to have trouble rendering the post if these files are absent though).
  
  I was unable to quickly find a convenient way to mimic how Graffiti generates these folders and files without resorting to writing a custom app of my own... but this is a simple test, so I deemed it not to be worth the effort.
  
  I personally would not want to deploy Graffiti on a large scale without a way to disable the creation of these files and folders. Strictly speaking, this feature is only necessary on IIS 6, or IIS 7 when using the classic asp.net pipeline. It would not be unreasonable to expect that Graffiti will support the IIS7 integrated pipeline in the future and allow the per-post folders and files feature to be disabled (please!). 

Graffiti Beta 1 - Review (part 1)

As some of you may have noticed, Reddnet.net got a face-lift last week.

I've migrated off of the BlogEngine code base (which I reviewed a while back) in favor of Telligent's new Graffiti CMS product. So far Graffiti is making me quite happy even though it is only in beta 1, but a few important features are still missing.

I thought I'd go ahead and do a review on it, but once I started writing I ended up with a lot to say. So I'll be splitting the review into a several parts over the next week or two.

So for the first part let me just tell you a little about Graffiti and Telligent in general.

Graffiti - It's not a blog exactly...

 Graffiti CMS is the newest product from Telligent, the folks that brought us Community Server.

 CMS is an industry standard acronym for "Content Management System", but with Graffiti the "CMS" part of this product's name supposedly stands for "Content Made Simple" instead. The dual meaning is quite deliberate, and Telligent is promoting Graffiti as a light-weight Content Management System even though they are playing around with the acronym.

 Graffiti is somewhere between a simple HTML content platform and a traditional CMS system.

 I should say up-font that I'm very skeptical of anything that calls itself a web based CMS system. The web itself IS a CMS system by design, and so the only real value anything can add to that is just gravy. CMS as a term is kinda hard to define. But generally CMS systems manifest by giving the site operator tools to organize, control, and search content once you've accumulated a metric ass-ton of it. Of course, any web platform has to provide this kind of functionality so generally when you hear someone talk about CMS they are either stating the obvious, or just talking out of their ass.

 But even though CMS is a flimsy marketing word there are a couple of things a good "CMS" will have to provide. The most important is tools for creating content that is of high quality. That means a simple and effective online content editor and usually also includes revision tracking, and content review and approval (work-flow) features. On the other end, a CMS should provide some mechanisms for organizing content. Usually that means categorization, labeling of some sort, reporting, and searching functionality. All this is necessary to keep old content from being buried under a mountain of newer stuff.

 As I said though, any decent web platform has to do this. The only difference between a "CMS" and an online catalog or online discussion forum is the kind of content being dealt with. In Graffiti's case, the content being managed is HTML "posts" like articles, blog posts, and similar.
 
 In beta 1, Graffiti has a good start on the publishing end of things. It provides the essential tools to easily and quickly creates content, and it presents that content in an effective way. It also presents content over RSS feeds as well as HTML web pages. It is also highly optimized to search engines and has a beautifully simple way of managing URLs automatically so they remain very human friendly.

 But the bulk of the heavy CMS features such as revisioning, review, and moderation are incomplete in beta 1. The searching, sorting, and categorization of content is present, but also not quite complete yet. There are a few rudimentary reports in beta 1 but presently this is limited mostly to summary traffic information... Basically beta 1's reporting amounts to a glorified "view counter", but the reports are very pretty.

 Fortunately for smaller organizations or casual bloggers, the heavy CMS stuff is unobtrusive, optional, and if used will add a lot of value without adding a lot of complexity.

 While Telligent is very careful to make sure everyone knows that Graffiti can do things other than blogs, the reality is that Graffiti's design is very much blog-inspired. The only real difference between Graffiti and most other blog kits is that Graffiti is a tad more polished, more flexible, and built to be extended in other directions. But the core is very bloggish.

 The light-weight CMS components will probably allow Graffiti a good bit of success in other markets such as for news and other "article" based sites as well as in product/service support applications as a knowledge base. I fully expect to see Graffiti add simple forums and wiki functionality very soon after the initial RTM release. If not, I expect the developer community will probably quickly move to supply such as add-ons.

 But the largest market for Graffiti will certainly be the bloggers. Beta 1 lacks some essential blog related functionality though. The biggest for me is support referral services and trackbacks. But those are promised for the final release, and  it already has rock solid posting and RSS support. 

About Telligent:

 I'm a huge fan of Telligent and I've been closely following their work since the company first formed a few years ago. If you were to ask me where I'd most likely seek employment if I were to relocate out of South Carolina, then I'd give you a list of companies with Telligent's name at the top. They are a great group of very talented people and they make some of the best code on the planet!

 On the flip side though, I've never been that big a fan of Telligent's flagship product, Community Server. CS is just too big, heavy, and purpose built for my tastes. CS is most certainly the best large scale forums and blog hosting platform on the market. CS does what it does very well, but it also sticks to its niche pretty tightly and is hard to adapt, especially in smaller environments.

 As a developer, I've always found extending and customizing CS to be... well... less than fun. It is quite painful and difficult to work with and requires a strong grasp of both .net as well as the CS product's internal architecture. CS has gotten much easier to develop against over the years though, but also during that time it has grown into a monstrous beast with about a dozen optional enterprise level add-ons. So while developing for CS may be easier than in the past, there is a lot more to develop against too. CS is almost so big that you could make an entire career out of just developing CS sites.... kinda like sharepoint, which is another product I'm not all that fond of --and for many of the same reasons.

The Waffle House Code Engine

I spend a lot of time at my local Waffle House writing code. I know, not the most up-scale choice, but unfortunately a necessary one considering the lack of other good options these days.

For those unfamiliar with it, Waffle House is a very large chain of greasy-spoon diners. It is nearly a national chain, but there are a few areas that haven't been graced with a waffle house yet but most places have something similar.

Waffle houses are very small restaurants seating around 25 people comfortably. They are usually located on major interstates or within major cities. They are also notoriously dirty places, with generally crappy food guaranteed to cause heat disease nearly instantly. The regular clients of such places range from retired old men to drunken motards with a lot of not much in-between.

On the plus side though, waffle houses are cheap places to eat (more or less), they serve great coffee. For their employees the house pays weekly, and they pay in cash too. This makes it the ideal job for drug-addicts, escaped convicts, illegal immigrants, and anyone whose idea of a "savings" account includes a mattress and some coupon books.

It wasn't always this way for me.

I used to hang out in higher-class establishments like IHOP or Denny's. Hey! I didn't say "high" class, just higher class than the awful waffle. But I'm a smoker, and I love sitting around with my laptop writing code while drinking coffee and smoking at 3am, and for that there just aren't many class-establishments to choose from.

It just works for me.

I've tried this at home, but all that mess about having to get up and make coffee, pour it myself when my cup runs empty... uug... and I don't smoke in the house so I have to actually go out to the garage when I want a smoke. Also, at home I have all my other stuff handy like the TV, a fast internet connection, and a lot of neglected chores that need doing.

So its easy to get distracted.

At the office, things are better. But there I have co-workers that need working with, my end-users have questions to ask, bugs to report, and just general problems that need solutions. Plus there is my boss who needs status updates and has new ideas and features to propose and some other projects to discuss. Then there are meetings and the occasional talking to the customer too.

In short:

if(codeAtHome == null && codeAtWork < optimal)
{
    DoWorkAtWaffleHouse();
}

So while I get code done at the office, and less often even at home, I still do my best work after-hours in places where there are people getting paid to look after me. Plus the staff makes good entertainment when you need to take a break from the screen for a bit.

But the anti-smoking movement is on its way, and I'm sure it wont last much longer, every year there is more and more political noise in the direction of banning smoking in all public places. At this point it may just be a matter of time. So once I either quit smoking (which I'm planning to do this year) or they finally ban it, the only real thing that changes is that I'll have more options for where to hang out.

Pack'n

Update: 6 months after buying the XD, I have replaced it with a Walther PPS 9mm. Find out why in my folllow-up post.

I was raised around guns. As a kid I lived on a small hobby farm in a rural part of South Carolina, and guns were just part of the scenery. As young as 6 or 7, I was using small rifles and shotguns. Not that I was much of a hunter or anything. Mostly the guns were for defense.

We raised cattle, and packs of wild dogs were still common in the area, so we carried weapons of some kind almost all the time in the woods or pasture. Usually we carried knifes, BB guns, pellet guns, or sometimes just big sticks. But if a pack of dogs had been sighted, suspicious strangers reported in the area, or a violent crime where the assailant was still at-large, we'd switch up to shotguns and rifles.

At least twice, having a gun on me in the woods saved my life. Once from a group of dogs and once from a very upset bull.

I have a healthy respect for the 2nd amendment and I am comfortable with guns, but I've not bothered to keep my own guns as an adult. I had several reasons including the high cost of weapons, but mostly it was just because I usually had roommates who owned plenty of guns. I just never felt the need to buy my own.

I'm not the kind to trust the police and government to protect me from danger. Police are great 10 minutes after the shit hits the fan, but they aren't likely to be handy when you are actually being assaulted, robbed, mugged, car-jacked, etc.  And governments are just as likely to be doing the assaulting as protecting anyone from it. Likewise though, I do have a good grasp of the actual rarity of random personal assaults. Even in the most dangerous of urban areas. most assaults are committed by people you already know and have pissed off. So I just make a habit not to know too many people likely to assault me, and avoid pissing off the rest so much. This greatly increases my odds.

Still though, I have a habit of being out late at night and hanging around slightly dangerous places. I also own my own house now and live alone most of the time, so there aren't roommates with guns around anymore. And that is why  I finally decided to buy a handgun, learn to shoot reasonably well, and get my conceal and carry permit. I have always carried a good knife on me, but I have no doubt that a knife would probably do little for me against a real threat.

Time to upgrade.

One thing that always bothered me about civilian gun owners is that they tend to get carried away. I don't know exactly what it is that makes an ordinary person become a gun-nut, but It does seem to be a very simple conversion for most people. They decide they want to buy a gun, and two weeks later they are at the gun-show picking up their 8th pistol and 3rd rifle and have most of their credit cards maxed out on accessories already. They'll go on for hours on end debating the merits of this vs. that weapon, talking technical details about the performance characteristics of different kinds of ammo, and worst of all --they spend an enormous amount of money buying weapons, accessories, and gear that they will likely never use other than recreationally.

I'm not a person who has a need to be a gun-collector, and I've never wanted to be a suburban gun-nut. But in becoming a gun owner, I see more clearly just how very easy it is to fall into becoming one. Guns are not simple, especially handguns. They come in an amazing array of sizes, shapes and calibers. Just deciding what to buy requires a significant investment in time and learning. And no handgun is good for everything. So it becomes very easy to have a conversation like this with yourself:

"I'll get the 5" 9mm semi-automatic for the shooting range. It is very accurate, holds a lot of ammo, and won't beat the shit out of me. It'd have similar enough recoil to my concealment weapon to make it good for practice."

"Then I can get the 6" .22 semi-automatic for the range because ammo is dirt cheap for those, almost no recoil so you can fire it all day long, plus they are just plain fun."

"But I still need a concealment weapon. For that I'd like to get the 3" sub-compact semi-auto .45 with double stack magazine. Big bullets for best chance to put down an attacker in one shot, double stacked for more bullets, and still short enough to fit in a concealment holster... more-or-less."

"But in the summer when I'm wearing thinner clothes, it would be easier to conceal that tiny little 9mm with the single stack magazine. It is really tiny, but with smaller bullets it holds as many shots as the double stack .45 and is a lot lighter. Of Course it will kick like a mule and be less accurate, so it's not that good at the range".

"OK, maybe I should go with a longer weapon. A 4" barrel will help with accuracy, even though it's harder to conceal, but that's OK I can go with a smaller caliber instead".

"Crap, what about brand. This one has a manual safety which I like, but it is double action on that first shot, which always throws off my aim. Maybe I should get the one with the trigger and grip safeties instead... I don't like that as much, but it has a single action first pull... which would be better in a crisis.

"What about a revolver. They are more reliable and a little simpler to use."

"Now, should I get all metal, or go with a polymer frame?"

"Fuck-it, I'll just buy ALL of them!"

  
So I can see how collectors and gun-nuts happen.

For the record, I decided on a 3" Sub-Compact XD .40 Caliber from Springfield Armory. It's a short and squat weapon, but still a tad on the thick and heavy side. For most people it probably wouldn’t matter, but I'm so thin that concealing a weapon of any kind is a tad tricky. But this one conceals reasonably well when I wear a coat or heavy shirt, and most of the time I'll probably just be in my laptop bag anyway (I rarely go anywhere without that at my side).

I stubbornly refuse to go crazy and become a total gun nut and collect a personal arsenal. But also, I do live in a two story house so I'll probably pick up a second handgun for up-stairs and eventually a slimmer 9mm carry weapon for those times when the XD is just a tad too big or when I'll be carrying for longer periods of time. Add that to a 12 Gauge shotgun, the ultimate home-defense weapon, and that should be about all I'll need. It's Four weapons, but still... most of my friends own twice that many or more...

Shit... OK, I'll admit it... I'm now officially a gun-nut. Damnit! But I promise not to get too carried away.

I'm not your prozac

Caress has started blogging her experience with AVM over at "I'm not your prozac". She wrote a lot of those posts while she was going through her procedures, but since I hadn't gotten off my lazy ass and put up her blog site (and she was a tad too busy to do it herself) she's didn't get them online until more recently. Most of her posts are back-dated to when she actually wrote them.

I've avoided blogging about the experience mostly because I was waiting on her to blog about it first, and also because AVM is too damned scary for me to blog well.

Akamai MSDN Download Manager sucks!

 Richard's Braindump: Problems with the new MSDN Download Manager

I agree with Richard, the new MSDN download manager is worse than horrid. I've ran into every one of the problems reported but these in particular really chap my ass:

• Close IE and the download manager closes. It asks you if you really want to close it, but it doesn't matter what you say... it will close anyway. When you re-open it, your download will be corrupt and you'll have to start over again from the beginning.
    
• It'll just randomly tell you it can't download because of persistent network problems... and you can't resume because your file is corrupt.
   
• If you have pop-up blocking on in IE 7 (which is the default) you might never get the download manager installed. It pops up a new browser window, which IE blocks, then the main page refreshes so fast you never see the notification that there was a blocked pop-up.

I have additional complaints too.

• Adding additional files to the download manager sometimes (and by "sometimes" I mean "nearly always") closes the download manager... which as we discussed already, corrupts any files it was already downloading.
   
• Manually pausing a download also usually corrupts it.
    
• When it asks if you want to start the download all over again, and you say yes, it usually can't... so you have to remove the file from the manager, go back to the MSDN page then re-click the link.... which will close your download manger and corrupt the rest of your files!

And even if they fix these major technical problems, there are other issues I have with it:

• Files are either downloading, or paused. There isn't a way to put them in any particular order in the queue and download them sequentially... so if you want to pull in 5 files, but you want 1 file immediately, then the rest to finish up whenever you have to pause all the downloads except the one, wait on it to finish, then manually resume the others.
   
• There doesn't appear to be any mechanisms in place to allow you to throttle how much bandwidth it uses.
   
• It allocates the disk space before the download starts. I HATE that. If you only have 1gig downloaded so far, it should not be taking up 4gig on my hard drive. When I'm low on disk space I can't start the download and have it coming down while I go and clean up some drive space. I have to clean up the space before I can start downloading. Minor issue, but still annoying.

Seriously, this thing is a total piece of shit... Why can't you just give me a damned link?

I have a download manager than works really good. I can control what files come down, in what order, how much bandwidth to use, and how many files to work on at a time (I use ReGet for those of you that may be curious).

Come on, just give me a regular old hyperlink or put it up on an FTP or something.

Considering that an MSDN Premium Team Suite subscription costs $10,000 up-front and around $3500/year for renewal fees,  it isn't too much to ask for is it?