Computer Cabal

SqlServer Maintenance: Keeping your indexes from becoming too fragmented

2012-01-11T15:02:00.000-08:00

A while back I was trying to solve a problem where a database server at work had been showing high CPU usage and over time it was slowly getting worse. After looking at the sql server profiler and filtering on queries that were taking longer than a second I found one of our queries was taking far longer than it should. I then went to Sql Server Management Studio to investigate the query with the "Include Actual Execution Plan" option turned on, when the results came back, the execution plan suggested I create some indexes on a few tables. That's fine, but once I looked into what these indexes were, they were the exact same indexes that I already had on the table! Turns out the existing indexes were badly fragmented, especially in the tables that have large amounts of inserts.

One way to quickly find the fragmentation of a particular index is to right click on the index withing Management Studio, click "Properties" and then click the "Fragmentation" page.

If you want to get an idea of how fragmented your indexes are for the whole database you can run this query:


SELECT a.object_id,
      b.name AS IndexName,
  a.avg_fragmentation_in_percent AS PercentFragment,
  a.fragment_count AS TotalFrags,
  a.avg_fragment_size_in_pages AS PagesPerFrag,
  a.page_count AS NumPages
FROM sys.dm_db_index_physical_stats(DB_ID(),
  NULL, NULL, NULL , NULL) AS a
INNER JOIN sys.indexes AS b ON a.object_id = b.object_id AND a.index_id = b.index_id
WHERE a.avg_fragmentation_in_percent > 5
ORDER BY avg_fragmentation_in_percent desc

One way to quickly fix this problem if you only have a few indexes with high fragmentation is to simply right click on the index and choose "Rebuild" or "Reorganize". The rule of thumb is to Reorganize an index when the fragmentation is between 5% and 30%, because it keeps the index available while it is reorganizing it. Above 30% fragmentation and the index should be rebuilt because the reorganization process for highly fragmented indexes takes too long.

This Microsoft article on Reorganizing and Rebuilding indexes has more information on the differences.

Four our purposes we found that just rebuilding the indexes instead of sometimes reorganizing them works well enough because it doesn't really take that long, even on our live system.

If you want to rebuild all indexes on a table you can issue the following TSql:


ALTER INDEX ALL ON <tablename>
REBUILD

However, we probably want to rebuild all indexes in our database and also schedule it to run periodically (once a week or once a day depending on the system load). Thankfully, this is very straight forward through the Maintenance Plan wizard:

From SQL Server Management Studio, open up the "Management" node, then right click on the "Maintenance Plans" node and select "Maintenance Plan Wizard" (if you get an error about enabling Agent XPs, just run the command here: Agent XPs Option).

Give the Maintenance Plan a name like "Rebuild Indexes", and set it to run on a schedule (maybe once a day or once a week).

Click Next and choose "Rebuild Index" on the Select Maintenance Tasks page.

On the next page, selecting the order of tasks, since there is only one task, click next.

Finally, define the options for the Rebuild Index Task, you can select only particular databases, and which objects you want, or as I have done simply "All Databases":

Click next and for the report options I don't really care about generating a report file so I uncheck that option and click next. You are done, click Finish and as long as you are running the process frequently enough you shouldn't have to worry about your indexes getting overly fragmented.

SSIS cannot use relative path for execute package task

2011-11-28T10:58:00.000-08:00

We just started using SqlServer Integration Services (SSIS) at work for our upcoming Data Warehousing project. So far learning my way around SSIS has been relatively painless, I ran into one issue though, there is no way to reference a relative path to dtsx packages (using the execute package task), you must use absolute paths. This doesn't work well for our team however, because many people have their working directories in different locations on their workstations. Apparently many people complain about this same issue and there is a request into microsoft to create a variable [System::PackageLocation] so that a relative path can be used. There are a couple of solutions offerered in other blog posts to get around this annoying problem: Flexible File System Deployment and Relative Paths in SSIS, however none of these solutions seemed very elegant or friendly so I've come up with my own work around using windows Junctions which is basically a symbolic link pointing one path to another path, and the tool is available as part of the windows sysinternals utilities package.

All you have to do is have your developers use the junction tool to agree upon a directory for the junction such as C:\SSIS-Working that points to the developers SSIS package path.
Simply run the command:


junction C:\SSIS-Working C:\<path to your working ssis package directory >

Then when you reference your package files within the execute package task, all you use is the C:\SSIS-Working\package-name.dtsx

Now all our developers can develop our SSIS packages and reference the absolute paths even when their working directories are in different locations.

ASP.NET Serialization Exception Crash, winsock and Nagle Algorithm

2011-11-16T11:58:00.000-08:00

Back in 2007 I wrote an article about our troubles with expect 100 continue and the nagle algorithm while performance testing our web application on windows 2003. Recently we started doing performance testing of our software on windows 2008 and after about an hour of modest traffic on our server the application would unexpectedly crash, the only error that we could find was in the event log:


Exception: System.Runtime.Serialization.SerializationException Message: Type 'System.Net.InternalException' in Assembly 'System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' is not marked as serializable.

This was a very odd error message because nowhere in our code do we do any binary serialization and it appeared that the InternalException itself was trying to be serialized. We added code to catch unhandled exceptions like we had back in the .NET 1.1 version of the code (we didn't bring our Unhandled Exception Module into the system when we rewrote the system in .NET 2.0), however that didn't seem to prevent the system from crashing when this error occoured and we didn't get much more useful information from the event log. There was something about the problem occouring at:


HttpWebRequest.CheckWriteSideResponseProcessing

Which is in the internal workings withing the .NET framework.

We engaged Microsoft support on this issue because it was beyond our expertise and with our MSDN subscription apparently we can submit four issues per year for free. After a few weeks of analyzing the problem and having MS support view dump files, they seem to have narrowed down the problem, apparently there is a race condition in the winsock api (which is what is used internally by our .NET Async Http Requests) where if you have Nagle turned off it can lead to this fatal error situation.

Since the last time we tested performance in our application we also switched the method our application was making requests from HTTP POST to HTTP GET, and it seems that we don't suffer the performance impacts using GET if we turn on nagle using Nagle and expect 100 continue (which is the default) as we did with POST, so removing the code:


System.Net.ServicePointManager.Expect100Continue = false;
System.Net.ServicePointManager.UseNagleAlgorithm = false;

fixes our issue. There is also another work around that Microsoft support has recommended we try which also appears to resolve the problem, and that is to set the option: alwaysUseCompletionPortsForConnect to true in the web.config file like this:


<configuration>
  <system.net>
    <settings>
      <socket alwaysUseCompletionPortsForConnect="true"/>
    </settings>

As described on this page: <socket> Element (Network Settings)

So thankfully it seems we can put this issue behind us and move onto other things, we have been really impressed with Microsoft support on this issue and are grateful for their help debugging this issue.

Finally, I'll mention another point of interest, that there is a setting in the Aspnet.config file for ASP.NET that makes .NET 2.0 behave like 1.1 in the sense that if an unhandled exception occurs it will not bring down the entire app pool. Microsoft warns against using because it can cause other problems - the application may leak resources and abandon locks. It is a nice option to have though so that in production this problem would not bring down the entire system while we tried to debug the issue. To do this, set the legacyUnhandledExceptionPolicy option to true as described in method 2 on this page: Unhandled exceptions cause ASP.NET-based applications to unexpectedly quit in the .NET Framework 2.0.

Wordpress sharedaddy jetpack plugin share button not working

2011-10-09T18:29:00.000-07:00

Today I was debugging a problem on my wordpress site: AllWebGaming.com where the sharedaddy share button was not expanding and would throw a javascript error. In Chrome I would get the following error:

Uncaught TypeError: Object #<Object> has no method 'swing'

In Firefox I was getting this error:

f.easing[e.animatedProperties[this.prop]] is not a function

when hovering over the share button.

After a little bit of searching, it looked like the culprit was a conflict with the easing jQuery plugin. Apparently the sharedaddy jetpack plugin uses this plugin and as it turns out the theme I'm using (Jarrah) also uses the easing jQuery plugin.

To resolve this, I simply went to edit the theme's header template file (header.php) to comment out the easing javascript reference (Appearance -> Editor -> Header).

I found this line in the header.php file:

And replaced it with html comments wrapped around the line:

I'm not sure if there is a better way to do this in Wordpress (like update the easing plugin for the jarrah theme), but this method worked for me to get the share button working again.

Volume low on videos played through Flash player

2011-09-23T16:03:00.000-07:00

I started having this problem on my work PC, every video I played though a browser that used Flash either couldn't be heard at all or was very feint, if I turned the volume up to max I could barely hear the sounds. Windows sounds played fine, everything through an app on my system worked fine, even sounds or videos played through html5 or silverlight work fine. I even uninstalled and reinstalled the flash player and still no volume coming out of my browser. This is one of the strangest issues I've ever seen, obviously my flash settings seem to have gotten corrupted somehow. I've even tried deleting all my saved data and settings within the flash player and upgrading drivers. I still can't solve the problem, if anyone knows of a reason why this would be, feel free to comment and help me out.

New Browser-Based Game List Site: All Web Gaming

2011-09-08T11:26:00.000-07:00

My latest project has been slowly building a Games directory/list site which is exclusively a list of Browser-Based games that use Flash, Silverlight or HTML5/Javascript. I've been trying to focus on games that might not otherwise get alot of attention and aren't already hosted on Game portal sites. The new website is called All Web Gaming, you can visit the site here:
AllWebGaming.com.

The site is still a work in progress, but it has enough games currently listed to be interesting for some people including great games like: Astriarch, Icycle, Black Market, and Flow.

Testing Text-To-Speech in .NET using C#: SAPI 5.1, 5.3, and 5.4

2011-08-15T11:27:00.000-07:00

Since 2003 I've been developing against the Microsoft Speech API (SAPI) when SAPI 5.1 was out. Over the years Microsoft has updated the version of the Speech API, with windows vista they shipped 5.3 and with windows 7 they shipped SAPI version 5.4, you can read more about SAPI and the different versions on the wikipedia page here:
Microsoft Speech API.

The Verbot product uses Text-to-Speech, originally SAPI 4 using MS Agents back in the Windows 2000/XP days, and eventually we added our own similar talking head technology that uses SAPI 5.

Recently we've run into problems where newer voice engines (ones that leverage SAPI 5.3 or SAPI 5.4) don't seem to work with our older SAPI 5.1 version of the SpeechLib (Microsoft Speech Object Library COM component). On my Windows 7 machine I built this very simple TTS Tester application that gets a list of the TTS Voices and puts them in a combobox and allows the user to send the voice engine some text to speak. Since I developed this on my Windows 7 machine, it is using SAPI 5.4 in order to test that version against the newer speech engines. Eventually I'd like to be able to have different versions of the tester which uses separate SAPI libraries in order to be able to better test against the speech engines out there (to see which engines work with different versions of the speech API). For now at this this simple test app may help other people develop against the Speech API and/or test their speech engines against the newer SAPI 5.4 version.

Here is the Source Code for the TTSTester Library:
TTSTester VS 2008 C# Code

Free Joomla 1.6 extensions for Social media sharing, Disqus, and Posting from RSS

2011-06-30T14:23:00.000-07:00

Lately I've been searching for good free Joomla 1.6 extensions/plugins that would allow me to post articles from an RSS feed, and add Disqus comments to the articles as well as popular social media bookmark/share/republish buttons such as twitter, facebook, and google +1.

For my first requirement, creating articles from an RSS feed, I'm using Feed Gator, which is pretty straightforward to setup, after installing you need to add your desired RSS feeds, categorize your feeds based on where you want them to show up in your Joomla site, and create "Category Blog" type menu items linking to your categories to show your articles. There is a way to setup Feed Gator to automatically import articles from your feeds through a cron job, but I haven't had time to figure out how set that up yet though my hosting provider.

The second extension I installed was the Disqus Plugin called RokComments, the main Disqus plugin listed in the Joomla extensions directory doesn't work with joomla version 1.6 so I found RokComments which seems to work fine except that I did get a 404 error after installing which was fixed by simply turning the developer mode on and then off again in the settings for the plugin.

Finally, I wanted to add a plugin to add social media buttons to allow my visitors to share (or repost/republish) an article on Facebook and twitter and like the pages on facebook, +1 them through Google +1, etc... It is surprising how long it took me to find a nice plugin for this, most of the social bookmarking plugins only worked for Joomla 1.5 and not 1.6 and many of them are not free. I finally found SocButtons which was easy to setup and configure, free, and works with Joomla 1.6!

Hosting multiple sites with iPage, installing Joomla CMS, and the "This site is under development" home.html page

2011-06-27T21:52:00.000-07:00

Choosing a Cheap Web Host

For quite some time I've been playing with different options for Free web hosting of my own domains, with google sites, google app engine, and google apps for your domain you can pretty easily setup a site and have it hosted for free (see my post on Configuring Google Apps to host your Domain for free from back in 2008 for more info).

Recently though, I bought a few more domains and I wanted a cheap host that could host all my domains under one account and I could use popular content management systems (CMS) such as Joomla, Wordpress, and Drupal.

There are countless hosts out there and many sites which compare and post reviews of the different hosts (such as webhostinggeeks.com).
Incidentally many of these hosts run on Linux/Unix infrastructures and offer PHP, Python, and MySQL (usually Windows based hosts offering .NET support are a bit more expensive, but winhost and m6 are a couple of cheap ones I found).

Installing Joomla on iPage

A couple of days ago I chose to go with iPage (iPage.com), they are currently having a $2.95 per month deal and I think their control panel is easier to use than the cPanel that many other hosts use.

Disclosure: I receive compensation from iPage if you choose to click the above link and sign up for the service. I am not affiliated with iPage and the opinions expressed here are my own.

Using iPage's custom control panel, installing Joomla is very straightforward using the SimpleScripts item from the control panel home. I selected Joomla from the simple scripts page, then clicked install, then selected my domain, choose advanced options and finally hit complete.

Fixing the home.html problem

However, after the install completes my home page gave an error saying: This site is under development, and looked like a typical domain host landing page with iframe content loaded by dnsnextgen or dsnextgen or something. After searching for awhile for the cause of this problem and how to resolve it, I found the following page: My web site is published but I can't see it, which is related to wordpress, but explained that iPage installs a home.html page by default and in the .htaccess file, home.html is loaded before index.php (which is what joomla uses), so either you can update the .htaccess file as explained in the above page, or simple ftp into your host and delete the home.html file.

Hosting multiple Joomla sites in iPage

Once I got over that hurdle everything went fine using Joomla on my new site, however I wanted to be able to host multiple sites through iPage, which according to their marketing material you are supposed to be able to do. When using the SimpleScripts installer you can install Joomla into whatever domain you choose, but by default it installs the software into the same (Root) directory. There was also no obvious way to setup the additional sites through the domain manager so that you could tell SimpleScripts you wanted to install Joomla into another folder.

I couldn't figure out a solution until I came across this post: How to host multiple sites on iPage. You need to setup a subdirectory pointer for the domain, so basically you just click Domain -> Domain Central from the main control panel in iPage, expand the '+' icon next to the domain you want to modify, select Subdirectory, and in the input box choose a directory name for your site. Once you go to install Joomla again from the SimpleScripts installer you'll notice that it will install into the selected subdirectory instead of the root! (Thanks Charles for the tip!)

SQL Server: Create a Maintenance Plan to keep the transaction log from growing too large

2011-06-17T11:09:00.000-07:00

This post is a follow-up to my post in April: Shrink Sql Server Transaction Log. If you database has lots of insert and update activities you might find yourself in the situation where the transaction log grows so fast that you want to schedule a process to run that will backup and shrink the transaction log files periodically. On one of our servers the transaction log had grown so large (~ 60 Gig) that when it needed to acquire more space in the transaction log it was taking a long time to reserve the space on the hard disk and during that time the database was unavailable (causing database connection timeout errors in our application).
One error we saw in the event log that was not very helpful to understand the problem was along the lines of:


A time-out occurred while waiting for buffer latch

There was one error that pointed us in the right direction:


Autogrow of file 'answerbase_log' in database 'answerbase' took 325418 milliseconds.  Consider using ALTER DATABASE to set a smaller FILEGROWTH for this file.

Even if you are doing a full or differential scheduled backup of your database (which you should be doing to an off-site location in case of disaster), you will find that your transaction logs will eventually grow to an unmanageable size and you will see failures as it tries to claim more space. To resolve this issue, we'll create a new Maintenance Plan task that will periodically backup our transaction log, and clean up the old transaction log backup files.

First open up SQL Server Management Studio and connect to your desired database. Next open up the Management -> Maintenance Plans folder nodes. Right click on Maintence Plans and choose "Maintenance Plan Wizard" and give the new maintenance plan a name like "Transaction Log Backup".

Click the "Change..." button under Schedule and choose a periodic schedule such as every day and every 15 minutes.

Click Ok to close out of the Job Schedule Properties dialog and click Next to continue the wizard. Then select "Back Up Database (Transaction Log)" on the Select Maintenance Tasks step.

Then click Next to continue, and then Next again to accept the maintenance task order. On the Define Back Up Database (Transaction Log) Task step, choose "All user Databases" and select the check-box "Create a sub-directory for each database" as well as "Verify backup integrity", all the other options can be left to the defaults.

Click Next and on the Select Report Options step, de-select the checkbox: "Write a report to a text file"

Finally, click Next and then Finish to complete the initial setup of the new Maintenance Plan. Once SQL Server is done creating the new Maintenance plan, click close to close the wizard. You may have to right click on the maintenance plans node in the object explorer to get your new maintenance plan to show up. Right click and select "Modify" on our new maintenance plan.

Next we want to add a new cleanup task to cleanup our old transaction log backup files. From the toolbox in the bottom left, locate the "Maintenance Cleanup Task" item under the General group and drag it to the edit area of the open Transaction Log Backup task. Then select the Back Up Database (Transaction Log) task and drag the green arrow from the bottom to the top of new Maintenance Cleanup Task item.

Right click on the new Maintenance cleanup task and choose "Edit...". In the Maintenance Cleanup Task dialog select the "Backup Files" radio button as well as the "Search folder and delete files based on an extension" radio button. For the Folder you will choose the location that the transaction log backups will be saved to, for my server in SQL Server 2008 the folder is:


C:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\Backup

Then type "trn" in the File extension text box (for the transaction log files), select the checkbox: "Include first-level subfolders" and finally I've chosen to delete files that are older than 3 days.

Select Ok to close the cleanup task window and finally click the save button in the top of the Sql Server Management Studio window to save our Maintenance plan. Our final plan should look like this:

With this Maintenance plan created and scheduled to run periodically, we will ensure that our transaction log doesn't grow significantly and we will still maintain our data integrity because we can keep the data recovery mode to FULL and not have to switch to simple mode which would prevent us from being able to restore to the exact point of the failure.

Astriarch 1.0.1 Released

2011-05-27T16:33:00.000-07:00

Astriarch: Ruler of the Stars version 1.0.1 has just been released.
Version 1.0.1 includes new features to improve playability. There are now planet strength indicators shown above each planet as well as a new feature to keep building the last ship built on each planet. The easy computer AI is now slightly less aggressive and a bug related to the AI build logic has been fixed.
Play Astriarch now at http://www.astriarch.com/

Astriarch - Ruler of the Stars is a turn-based single player space strategy game implemented in Silverlight. Build planetary improvements and star ships to capture planets and defeat your enemies. Your ultimate goal is to become the master of the known universe, and earn the title of Astriarch!

Developer Perspective: What questions to ask about a Company at a job interview?

2011-05-09T16:49:00.000-07:00

When interviewing for a new job, it is important to remember that the interview process is a two-way street. It is important to remember that it is as much about answering the questions of your perspective employer as it is about making sure your new job is right for you.

I've assembled a list of questions you might consider asking either before, during or after an intervew:

How long has the company been in business? How many employees are in the company and how many work at this office?
How many departments have developers? How many developers are there total and how many developers are in a team?
What systems are maintained by the development teams now?
What are the primary technologies and programming languages used?
What kinds of projects are underway now?
Where is the technology headed long-term at the company?
What is the revenue and profit of the company? How stable is the position, are you hiring for other positions?
What is the development and release process? Is there a QA department? Do you have tools to facilitate the development and testing process such as source control, automated test, and continuous integration tools?
What IDE do developers use? What 3rd party libraries or platforms does development use?
What development methodologies are used? (i.e. SCRUM/Agile)
What position do you see me filling? What would my day to day responsibilities be?

This is a short list, but it should get you thinking about what types of questions you'd like to have answered about a company before accepting an offer. Remember, the interview process is about making sure the company is a good fit for you as much as making sure you are a good fit for the company.

What other questions are good to ask of a perspective employer?

Silverlight: Don't use MediaElement and MessageBox

2011-05-09T12:29:00.000-07:00

When adding music to my game, I came across a severe problem after adding the MediaElement to my silverlight game in order to play the sounds. The problem is that in some areas of my game I was using MessageBox.Show to show little alerts to the user. However, once one of the MessageBoxes popped up, all hell broke loose, the computer sized up and I couldn't even bring up the task manager in order to kill my browser process! Many times during debugging I had to hard-restart my system because of this. There must be something about how the MessageBox blocks the background thread of the MediaElement which causes this severe crash. I didn't look into it enough to know what the exact cause is, but let this be a warning, if you are using the MediaElement, make sure you don't have any calls to MessageBox.Show, instead use the ClientWindow for these types of notifications.

Astriarch Space Strategy game now Gold

2011-05-05T11:30:00.000-07:00

Astriarch - Ruler of the Stars version 1.0 has just been released. A stellar ambient soundtrack has been added and other game balance improvements have been made. Special thanks to Resonant for composing the excellent music.

Play Astriarch now at http://www.astriarch.com/

Astriarch - Ruler of the Stars is a turn-based single player space strategy game implemented in Silverlight. Build planetary improvements and star ships to capture planets and defeat your enemies. Your ultimate goal is to become the master of the known universe, and earn the title of Astriarch!

Shrink Sql Server Transaction Log

2011-04-20T15:14:00.000-07:00

If you've run out of space on a machine running SqlServer 2000, 2005 or 2008, you may have run into a problem where the database transaction log file has grown too large and you need to reduce the size.

For Sql Server 2000 and Sql Server 2005, use the folowing command:


use <dbname>

backup log <dbname> with truncate_only

dbcc shrinkfile (<dbname>_log, 1)

Replace <dbname> with the name of your database.

For Sql Server 2008, the procedure has changed a bit:


Use <dbname> 
GO
Alter Database <dbname> Set Recovery Simple
GO

Alter Database <dbname> Set Recovery Full
GO

DBCC SHRINKFILE (<dbname>_log, 1)
GO

Again, replace <dbname> with the name of your database.

Hope this helps, seems there are many articles about reducing or shrinking your database log files, however there seems to be a lack of succinct examples out there.

Google Closure Compiler using ADVANCED_OPTIMIZATIONS to minify and obfuscate JavaScript

2011-04-17T15:20:00.000-07:00

Lately I've been working with the Google closure compiler in order to minify and obfuscate my javascript code. There are two main routes to go when using the Google closure compiler, SIMPLE_OPTIMIZATIONS or ADVANCED_OPTIMIZATIONS. Simple optimizations will allow you to easily concat and compress your javascript by removing whitespace and comments, but it won't do much else. With the Advanced optimizations option the closure compiler will rename your variables and functions as well as restructure your javascript so that it is more efficient to download (smaller size) and executes faster. All this power doesn't come at a price however, there are quite a few things to consider when using the ADVANCED_OPTIMIZATIONS mode.

First off, advanced optimizations will remove functions that the compiler thinks aren't called, so if you have javascript code within other files that aren't compiled you'll want to move those calls into the list of compiled files, or use the method described under the section Solution: "Export the Symbols You Want to Keep" in the document: What to Watch Out for When Using ADVANCED_OPTIMIZATIONS

Another problem to watch out for when using the google closure compiler ADVANCED_OPTIMIZATIONS mode is calls to 3rd party libraries. Since my project uses jQuery, I don't want the closure compiler to muck with my jQuery function calls. To get around this problem, you must specify an 'extern' file for the compiler, this tells the compiler that any function specified within the extern file must be preserved in the compressed application files. Google provides some common extern files to use such as externs for jQuery versions: closure compiler externs.

If your javascript objects reference themselves using the 'this' keyword, you will likely see the following warning when compiling your javascript:

WARNING - dangerous use of the global this object

Because the closure compiler will optimize your javascript for performance it will do what is refered to as "namespace flattening" and as a result 'this' will no longer point to your object, but instead the global window object. To work around this problem, the closure compiler expects you to annotate your javascript using the jsDoc syntax. Read Annotating JavaScript for the Closure Compiler for all annotation options and syntax. Lets take a simple example and describe how it should be annotated to avoid this problem.


Foo = function() {
this.member = {'count':0};
}
Foo.prototype.doSomething = function() {
this.member.count++;
}

Without annotation the closure compiler will make doSomething a global function (not a prototype function of the Foo object) and the 'this' reference inside doSomething will cause an error because it will now be pointing to the global window object. To fix this let's now annotate the code above so that the closure compiler won't cause this problem.


/**
* Foo is my constructor
* @constructor
*/
Foo = function() {
this.member = {};
}

/**
* does something
* @this {Foo}
*/
Foo.prototype.doSomething = function() {
this.member.count++;
}

The key here is the @this {Foo} specification, this tells the compiler that 'this' is an instance of the Foo class and it will not flatten the namespace.

Yet another problem you might come across when using the advanced options in the compiler is: Using string names to refer to object properties, described under the section: Restrictions for ADVANCED_OPTIMIZATIONS in the page: Understanding the Restrictions Imposed by the Closure Compiler
. Basically, because the closure compiler will not change anything within a string in your javascript, if you've defined an object like this:


var Astriarch = {'Version': '0.9.7', 'GameModel':null};

Note the quotes around the property names, let's say you then reference this property without the quotes, like this:


var version = Astriarch.Version;

The closure compiler will change Astriarch.GameModel because it is not quoted but will not modify your original object definition. To fix this, just make sure you always either use no quotes in your object definitions, or you reference your properties using the quotes. Like this:


var Astriarch = {Version: '0.9.7', GameModel:null};


var version = Astriarch['Version'];

To make this process streamlined I've create a batch (.bat) file to run to compile all my scripts using the closure compiler, this is the command line within the batch file:

java -jar compiler.jar --compilation_level ADVANCED_OPTIMIZATIONS --formatting PRETTY_PRINT --js_output_file astriarch.js ^
--externs jquery-1.4.4.externs.js ^
--js "./astriarch1.js" ^
--js "./astriarch2.js" ^
2> output-debug.txt

Note that for now I'm using the --formatting PRETTY_PRINT option to make it easier for me to debug problems caused by the closure compiler, once I get all those fixed I'll remove PRETTY_PRINT because it won't add additional newlines and whitespace. Also, the last line: '2> output-debug.txt' redirects the standard output from the closure compiler to a file, otherwise my thousands of warnings and errors would just be printed to the cmd console window and it would be impossible to see them all.

I'm continuing working with problems caused in my application by the Advanced compilation mode for the google closure compiler, hopefully I'll be able to fix all the remaining problems and in the end have a nice minified and obfuscated javascript file for the html5 version of Astriarch.

jQuery UI Listbox widget plugin

2011-03-10T22:44:00.000-08:00

In one of my previous posts: Porting Silverlight App to javascript using HTML5 Canvas tag, I mentioned that I needed a widget similar to Silverlight's listbox control and that if I had some time I would release my jQuery UI listbox widget. Well, I had a little extra time and I was getting impatient waiting for jQuery UI 1.9 to come out so I went ahead and published my current code as well as a simple example to github:
jQuery UI Listbox widget plugin.

The simple library uses the jQuery UI Menu component (which is used in the autocomplete widget), and is styled according to your jQuery UI Theme. Since the menu component will not be officially released by the jQuery UI team until version 1.9, there maybe some changes required in my plugin once jQuery UI 1.9 has been released. I wanted to make it available to anyone that was interested before that though since I might get some feedback in the meantime.

Stratiscape multi layer HTML5 canvas library on GitHub

2011-02-15T12:01:00.000-08:00

I've just published (pushed) a simple sample using my Stratiscape canvas library to GitHub. The sample displays an animated pinwheel drawn using the canvas javascript functions: createRadialGradient and drawImage. I also added a foreground layer of dandelion seeds floating around in front of the rotating pinwheel, a bit of wind and random movement and rotation and you have a captivating screensaver-esq Pinwheel and Dandelion animated canvas. I also added buttons to change the wind direction and speed to show how Stratiscape can be used to capture mouse events and allow the user to interact with the canvas. One thing that I'm not happy with is that in order to get the pinwheel effect I wanted I had to use a rotating image which isn't consistant between firefox, chrome and safari (for some reason my radial gradients on safari are a bit darker, and on chrome the image rotation is off during portions of the pinwheel spin). I was trying to think of a way to use the globalCompositeOperation function to get the infinite layered effect I wanted on the radial gradients, however I couldn't figure out how it would work in the short time I looked at it.

Porting Silverlight App to javascript using HTML5 Canvas tag

2011-02-08T10:12:00.000-08:00

I'm well underway in converting my Silverlight game: Astriarch - Ruler of the Stars to javascript using jQuery and the HTML5 Canvas tag. I was playing around with the Windows Phone 7 SDK, thinking that it should be relatively easy to port my Silverlight application to windows phone 7, however it wasn't as straightforward as I would have hoped and since the app was built for 800x600, scalaing it down for the windows phone 7 size was not going to look all that great.

Instead I decided to port the application to HTML5 using javascript and the canvas tag, this will allow the game to be played on a much wider range of devices (tablets and phones), and I may create targeted android and iPhone applications using the new HTML5 codebase.

I looked at quite a few different javascript canvas libraries, especially ones that allow multiple layers (easelJS, Artisan JS, Priit Kallas' HTML Canvas Javascript Library, Michael Camden's layered-canvas-library, and MooTools Canvas Library) however many of them were far more complex than I needed so I created my own simple canvas library. I'm calling the library: Stratiscape, it's still being developed and currently it's very much alpha code, but it is functional. I'll be creating some simple examples of it's usage in the future and working out any kinks I find.

I'm using jQuery UI for many of the specialized UI components to replace their counterparts in Silverlight such as Button, Slider, and Dialog. One control that I needed that jQuery UI doesn't currently have is the listbox, however I found that there is an undocumented/unreleased widget in jQuery UI called menu which the auto-complete leverages. I've been able to wrap the menu widget in my own javascript listbox control and it works very nice as a replacement for the silverlight listbox control. When I have some time to clean that up I may decide to release the listbox as well so that other people may benefit from it (I'll probably wait for jQuery UI 1.9 to be released because they plan on officially releasing and supporting the menu widget as a separate component).

Porting the Astriarch game is currently taking up most of my free time, but when that's done I'll be able to go back and clean up and release some of these other libraries I've created in the process.

Internet Explorer Developer Tools Dissapear

2011-01-07T16:00:00.000-08:00

I've been using the IE Developer Tools in Internet Explorer 8 exensively lately because of problems I've been trying to solve, however the other day I went to open the developer tools window (or click F12) and it showed the developer tools window at in the Windows task bar, however hovering over it in windows 7 just shows a blank window and I don't see it anywhere on my screen. Tons of people report problems with the developer tools, (it does seem like a very fragile extension), and I tried a bunch of the solutions suggested with no luck. Finally I found one post that suggested that the developer tools are most likely off-screen. If you select the window and hit windowskey-left arrow or windowskey-right arrow the developer tools reappears!

IE Quirks Mode and Font-Size:small

2011-01-06T12:29:00.000-08:00

At work we have used IE Quirks mode to get around some of the nightmares of the past trying to layout html elements in Internet Explorer, the latest hassle I had to overcome was an issue where it seemed that IE in Quirks mode would not respect my Font-Size:small css specification. Finally my co-worker suggested I try a percentage adjustment so I changed it to Font-Size:130% and it worked! I'm hoping that soon we can switch away from using Quirks mode and we can move forward to standards mode, but since we've developed our application from the ground-up using quirks mode there are countless problems when we switch to standards, so for now we'll just have to keep fighting with these hassles caused by the horrors of using Quirks mode.

Upgraded to Xcode 3.2.5, project using ios sdk 4.1 throws errors in AvailabilityInternal.h

2010-12-21T19:42:00.000-08:00

I opened an old Xcode project today and was surprised when there were 37 build errors in my project that used to compile fine, that's when I remembered I had upgraded Xcode somewhat recently so it was probably something going on in the ios framework. The build errors were inside one of Apple's internal files: AvailabilityInternal.h where it was checking the __IPHONE_OS_VERSION_MIN_REQUIRED macro, the error was something like: .../usr/include/AvailabilityInternal.h:56:42: error: operator '<' has no left operandAfter poking around in the project options (Project -> Edit Project Settings from the menu), I simply had to change the Base SDK option from iphonesimulator4.1 to Latest iOS (4.2). Once I did that everything was back to normal, an easy solution to a cryptic problem that no one else seemed to report so I figured I'd write a quick entry about it.

Discovered NSPredicate for easily filtering items in an array in Objective-C

2010-12-19T17:48:00.000-08:00

I've been learning Objective-C lately and building some iPhone and iPad applications in the process. Today I needed to filter out items in an array matching a search term from a UISearchBar. I was about to write a simple for loop to iterate through the items, check if a particular member of the objects of the array matched to my search string and then add them to a new filtered array. That's when I noticed the filterUsingPredicate method on the NSMutableArray object, with a nice SQLite-like syntax I could filter my array to just the items I was interested in! It reminds me of the power of the LINQ features provided in more recent versions of Microsoft's .NET framework. This saved me lots of tedious code, so with just a few lines I can filter out my arrays exactly how I want.


- (void)searchBarSearchButtonClicked:(UISearchBar *)searchBar
{
  NSPredicate *predicate = [NSPredicate predicateWithFormat:@"(memberName LIKE[cd] %@)", [[searchBar text] stringByAppendingString:@"*"]];
  //arrFull is my original unfiltered array
  NSArray *arrFiltered = [arrFull filteredArrayUsingPredicate:predicate];
  //...now any processing for arrFiltered 
}

Sometimes it doesn't hurt to check the documentation before doing something brute force, in my case it was an accident because I happened to be looking there anyways, but it saved me lots of lines and now I end up with a much more maintainable and elegant bit of code!

Mute the Microphone from C# on Windows

2010-11-12T10:44:00.000-08:00

I've recently had the need to Mute the Microphone device in Windows from code. There are quite a few sources for muting the microphone from c++ code, but I needed to be able to mute the microphone from .Net using C# managed wrappers.

In Windows XP and older windows operating systems you can use Microsoft's MCI (Media Control Interface) to control Audio hardware such as the microphone and thanks to this post on stackoverflow: How to mute the microphone c# I was able to figure out how to use Gustavo Franco's MixerNative AudioLib source code to mute the microphone. Gustavo's source code is a full mixer application and was overkill since I only needed to mute the microphone, so I was able to use pieces of his code and some of the c++ code in the stackoverflow post to get directly to muting the microphone.

After figuring all this out, I realized that this code didn't work on Windows 7! Turns out that starting with Windows Vista, Microsoft created a new Core Audio API that is supposed to make interfacing with audio hardware easier for developers. Since again all these api's are natively accessible from c++, and since there are no nice and easy ways to call them from .NET in C#, developers have to write there own managed wrappers to invoke the functions using the System.Runtime.InteropServices namespace.

Thankfully Ray Molenkamp wrote a nice C# managed wrapper for accessing the Vista Core Audio API which is available over at the Code Project. I tried taking Ray's library apart as well like I did with Gustavo's, partially so I could better understand the code, and partially because his library does lots more than I needed simply to mute the microphone. However after piecing together the parts of his code I needed to just mute the microphone it wasn't working quite right so I just gave up and my library uses his CoreAudioApi outright.

Since this was a bit of a pain to get working and piece all together I've decided to make the library I wrote available which allows you to mute the Microphone on Windows XP, Vista and Windows 7 (and possibly older windows operating systems too, I haven't tested it on 2000 or anything else).

To use the library simply instantiate an object instance of the WindowsMicrophoneMuteLibrary.WindowsMicMute class:


WindowsMicrophoneMuteLibrary.WindowsMicMute micMute = new WindowsMicrophoneMuteLibrary.WindowsMicMute();

Then simply call MuteMic() and UnMuteMic() as desired on the WindowsMicMute object:


micMute.MuteMic();
micMute.UnMuteMic();

You can download my WindowsMicrophoneMuteLibrary Source Code from here:
WindowsMicrophoneMute.zip

If you just want the dll binary files, you can download those here:
WindowsMicrophoneMuteLibraryBinaries.zip

New version released of DupeMan - The Duplicate File Manager

2010-11-06T20:43:00.000-07:00

I just released a new version of the Duplicate File management program: DupeMan. It has been quite awhile since I've released any updates, especially since I've been busy working on my first game: Astriarch - Ruler of the Stars. The changes I've made to DupeMan are mostly threading improvements so that there is the application is usable while indices are loaded and being processed. This includes a status area progress bar so that you can see how long indexing your file, loading index files, and checking for duplicates is going to take. I've also moved the page for the Duplicate file manager from my personal site to my MasteredSoftware.com site, you can download the new version here:
http://www.masteredsoftware.com/software/dupeman

Astriarch - Ruler of the Stars Browser based Space Strategy Game

2010-10-12T11:12:00.000-07:00

I've finished and posted the tutorial videos for the Free Space Strategy Game that runs in your browser (Silverlight plugin required): Astriarch - Ruler of the Stars

The tutorial videos show you how to get started playing Astriarch, as well as some tips on how to beat the computer. Since I couldn't edit the tutorial down to the 15 minute limit of youtube, I had to post it in two parts.

Also since I posted the tutorial I've made some improvements to the game including adding starship advantages, each starship type gets a bonus when attacking a starship type it has an advantage against.

I'm still trying to get a soundtrack in the game and some sound effects to make the game a bit more engaging, once I'm done with that I'll try to promote the game on some of the Silverlight game portals.

Error Setting up mirroring for Sql Server 2008

2010-10-06T18:05:00.000-07:00

I've been setting up SQL Server Mirroring for High Availability recently, and thanks to Edwin Sarmiento's blog post on Implementing Database Mirroring in SQL Server 2005 across domains, the process has been relatively easy in the test lab.

However, upon following the same steps in production I ran into a few problems.

Since in my environment I do not have a domain controller so the fragile part of this whole process is logins and passwords between the two sytems.

The first problem I ran into was after I had setup the endpoints and certificates and logins, etc... the database mirroring monitor would show an error "can't connect" and when I looked in the windows event log I saw this error:
SSPI handshake failed with error code 0x8009030c while establishing a connection with integrated security; the connection has been closed.

Which was a little bit helpful, but google was not in this regard, this error apparently shows up everywhere and I couldn't find any diffinitive answers.

So one thing I checked was the client protocols in the Sql Server Configuration Manager, I've had problems before connecting to remote servers that have been resolved by enabling all the protocols (such as TCP/IP and Named Pipes).

After this, I restarted both the primary and mirror servers to make sure the settings in the configuration manager took.

I was still however getting errors in the event logs and every time I opened the database mirroring monitor I would still get the error showing unable to connect to the other server. The strange thing was that on the principal server when I right-clicked the database and selected "Tasks -> Mirror..." it showed in the status area: "Synchronized: the databases are fully synchronized".

Finally I figured out that the problem was the credentials the database mirroring monitor was using to connect to the principal and mirror servers which you can configure by right clicking on the root node of the Database Mirroring Monitor and selecting "Manage Server Instance Connections".

From here you can click the edit button to give the sa (or other sysadmin user) for the remote machine.

Another problem I had at this point was the server names, after sql server was installed the server names were changed which also made sql server very grumpy, because it was thinking the machine name was the old machine name and said this message to the effect of: "SQL Server replication requires the actual server name to make a connection to the server. Connection through a server alias, IP address, or another other alternate name are not supported..." I have no idea what the magical combination of things I had to do to get past this error, trying every combination of names and just letting it fail, removing the registration, and re-registering the mirrored database I eventually got it to work, this was probably the most frustrating part...
If you can't get this to work after renaming the machine, you can try this process:


EXEC sp_dropserver 'oldName'
GO

EXEC sp_addserver 'newName', 'local'
GO

Restart your SQL Server service and then try the following sql:


SELECT @@SERVERNAME
GO

After setting this up the Database Mirroring Monitor shows successful on both machines.

System Master is now Astriarch - Ruler of the Stars

2010-09-23T21:59:00.000-07:00

After announcing the space strategy game I've been developing in my last post, I've been wanting to rename System Master to something a little more descriptive and unique. After playing with many ideas and thanks to wiktionary.org for helping me with entomology, greek and latin, I finally came up with a name that I was happy with: Astriarch - Ruler of the Stars.

You can see the website for the renamed game at the domain: Astriarch.com

I'm currently working on a tutorial video right now that should help people learn how to play. I'll post that as soon as I'm done.

System Master - Space Strategy Game in Silverlight

2010-09-07T17:29:00.000-07:00

For quite some time I've wanted to build video games, and my favorite video games to play have always been space themed games that involve strategy and/or civilization building.

Over the last few months in my spare time I decided to play around with Microsoft's Silverlight development platform and I decided that what better way to play around with Silverlight than to build a space strategy game! For a long time I had considered building a game using Flash because everyone has flash installed, but since I don't know the Flash development tools very well and I'm very comfortable with C#, I figured Silverlight was a good fit.

As of today, System Master is released as a public beta, it is playable yet it desperately needs music and sound effects. In the future I may decide to port the game to the iPad, but silverlight was a great platform to get started quickly and play with ideas.

I also intend to publish some tutorial videos to get people started because the in-game help is almost non-existant.

Here is a list of old-school games that gave me inspiration when developing my space strategy game - System Master:

Master of Orion 2
Stellar Conflict (space strategy game for the Commodore Amiga)
Star Control
Star Reach - also known as Space Federation (another great space game for DOS, arcade and strategy)

Please give System Master a go and let me know any comments you have!

Deployment of Crystal Reports with Sub Reports on Multiple Servers and issues with ApplyLogOnInfo

2010-08-03T11:01:00.000-07:00

We have been using Jan Schreuder's Crystal Reports helper class to open and display reports within our .NET web application. Jan's library makes it easy to set parameters programmatically from C# and takes care of some of the grunt work setting up credentials for the report so that the database connection can be changed at runtime.

For quite some time we have used the library without problem, but have never written a report containing a subreport. Now that we have reports that have sub-reports inside them with different commands than the main report, we started running into problems. After lots of debugging I narrowed it down to the call to "ApplyLogOnInfo" inside Jan's AssignTableConnection function, as far as I can tell the "ApplyLogOnInfo" is supposed to set the credentials for the table connection passed in, however it seemed to also override the command for the subreport so that the sub-report's command ended up being the same command as for the main report. When deploying the report to different servers we would see duplicated data within the report.

I commented out the call to ApplyLogOnInfo and instead just manually update the settings for the table's LogOnInfo's properties.
Here is a listing of my new AssignTableConnection function:


private static void AssignTableConnection(CrystalDecisions.CrystalReports.Engine.Table table, ConnectionInfo connection)
  {
   // Cache the logon info block
   TableLogOnInfo logOnInfo = table.LogOnInfo;

            connection.Type = logOnInfo.ConnectionInfo.Type;
   
            // Set the connection
   logOnInfo.ConnectionInfo = connection;

   // Apply the connection to the table!
            //table.ApplyLogOnInfo(logOnInfo);//commented out Aug 2nd 2010 MPP for subreports seems that this actually overrides the command so if you have a different command in the subreport it uses the command from the main report

            table.LogOnInfo.ConnectionInfo.DatabaseName = connection.DatabaseName;
            table.LogOnInfo.ConnectionInfo.ServerName = connection.ServerName;
            table.LogOnInfo.ConnectionInfo.UserID = connection.UserID;
            table.LogOnInfo.ConnectionInfo.Password = connection.Password;
            table.LogOnInfo.ConnectionInfo.Type = connection.Type;
  }

Another change we have made in our library that we've had for quite some time is a call to SetDatabaseLogon on the _reportDocument and subDocument objects within Jan's AssignConnection function. Here is that full function as well for completeness:


private void AssignConnection()
  {
   ConnectionInfo connection = new ConnectionInfo();

   connection.DatabaseName = _datebaseName;
   connection.ServerName = _serverName;
            if (_integratedSecurity)
            {
                connection.IntegratedSecurity = _integratedSecurity;
            }
            else
            {
                connection.UserID = _userId;
                connection.Password = _password;
            }
            
            

   // First we assign the connection to all tables in the main report
   //
   foreach (CrystalDecisions.CrystalReports.Engine.Table table in _reportDocument.Database.Tables)
   {
    AssignTableConnection(table, connection);
   }

   // Now loop through all the sections and its objects to do the same for the subreports
   //
   foreach (CrystalDecisions.CrystalReports.Engine.Section section in _reportDocument.ReportDefinition.Sections)
   {
    // In each section we need to loop through all the reporting objects
    foreach (CrystalDecisions.CrystalReports.Engine.ReportObject reportObject in section.ReportObjects)
    {
     if (reportObject.Kind == ReportObjectKind.SubreportObject)
     {
      SubreportObject subReport = (SubreportObject)reportObject;
      ReportDocument  subDocument = subReport.OpenSubreport(subReport.SubreportName);
                        
      foreach (CrystalDecisions.CrystalReports.Engine.Table table in subDocument.Database.Tables)
      {
       AssignTableConnection(table, connection);
      }

                        subDocument.SetDatabaseLogon(connection.UserID, connection.Password, connection.ServerName, connection.DatabaseName);
     }
    }
   }
            _reportDocument.SetDatabaseLogon(connection.UserID, connection.Password, connection.ServerName, connection.DatabaseName);
  }

That should help anyone using Jan's library with reports containing sub-reports and trying to set the connection info at runtime. Overall, I'm very happy with the CrystalHelper library and I really appreciate the time Jan has put into developing it.

Silverlight: Modify BitmapImage and Load Image Synchronously

2010-06-08T09:47:00.000-07:00

I am currently working on a simple Space Strategy game in Silverlight (I'll post a link when it's closer to being done). I have some images which I had the need to change the color of which I was able to do without too much trouble with the following function (thanks to the WriteableBitmap object):


private static void ChangeImageColor(Image img, Color colorNew)
        {
           
            WriteableBitmap bitmap = new WriteableBitmap(img, null);
            for (int y = 0; y < bitmap.PixelHeight; y++)
            {
                for (int x = 0; x < bitmap.PixelWidth; x++)
                {
                    int pixelLocation = bitmap.PixelWidth * y + x;
                    int pixel = bitmap.Pixels[pixelLocation];

                    byte[] pixelBytes = BitConverter.GetBytes(pixel);

                    if (pixelBytes[3] != 0)//if the pixel is non-transparent?
                    {
                        pixelBytes[0] = colorNew.B;//b
                        pixelBytes[1] = colorNew.G;//g
                        pixelBytes[2] = colorNew.R;//r

                        bitmap.Pixels[pixelLocation] = BitConverter.ToInt32(pixelBytes, 0);
                    }
                }
            }
            img.Source = bitmap;
        }

Basically I have images with transparent backgrounds and a single other color I wanted to change, so the above code just changes any non-transparet pixels to the desired color. Which seems to work fine, the part that I fought with for awhile was that I couldn't get the BitmapImage to load synchronously so that the pixels were immediately available, because when I just used the BitmapImage constructor which takes a Uri object:


BitmapImage bi = new BitmapImage(new Uri(relativePath, UriKind.Relative));

the bitmap pixels were all zero.

Then I just tried to use a FileStream to open the file but I kept getting access denied errors, so I found the following page:
Improving Silverlight Application Performance

Which pointed to the following lines of code:


StreamResourceInfo photoStream = Application.GetResourceStream(photosDownloaded, new Uri(photoToGrab, UriKind.Relative));

BitmapImage bitmap = new BitmapImage();
 
bitmap.SetSource(photoStream.Stream);

Which seemed to do the trick to get the image to load synchronously and not give me the access denied errors.

So here is my second function to give a new image brush with the updated (new colorized) image:


private static ImageBrush GetColorChangedImageBrush(string uriRelativePath, Color colorNew)
        {
            BitmapImage bi = new BitmapImage();
            bi.CreateOptions = BitmapCreateOptions.None;

            System.Windows.Resources.StreamResourceInfo bmpStream = Application.GetResourceStream(new Uri(@"SystemMaster;component/" + uriRelativePath, UriKind.Relative));
            bi.SetSource(bmpStream.Stream);

            Image i = new Image();
            i.Stretch = Stretch.None;
            i.Source = bi;
           

            WriteableBitmap wb = new WriteableBitmap(i, null);
    
            //change the non-transparent pixels to the owner color
            DrawnPlanet.ChangeImageColor(i, colorNew);

            ImageBrush ib = new ImageBrush();
            ib.Stretch = Stretch.None;
            ib.ImageSource = i.Source;

            return ib;
        }

Note that in the uri path I need to give the path to the qualified resource name with the "ApplicationName;component/" prefix ("SystemMaster;component/").

Hope this helps someone else trying to do a similar with Silverlight.

Crystal Reports Chart and Graph Images don't show on Windows Server 2008

2010-03-12T16:07:00.000-08:00

At work we use the Crystal Reports basic for Visual Studio 2008 (version 10.5) to allow our users to view reports, everything was working fine while we developed in XP and were using Windows Server 2003, except when we tried to move to Windows Server 2008 and the images served by the CrystalImageHandler.aspx http handler wasn't working for whatever reason and the dynamically generated images for the charts and graps in Crystal Reports were showing a red X (or just the alt text of "Image").

After much searching for the resolution of this problem, and lots of tweaks to the web.config and the temp folder security (which didn't help), I found one page which pointed me in the right direction:
http://forums.sdn.sap.com/thread.jspa?threadID=1338110

Basically I created a new dummy page and dragged the CrystalReportViewer control onto it, which auto-updated my web.config with a couple of items I was missing apparently. I already had the section under the httpHandlers node for the CrystalImageHandler.aspx:


<httpHandlers>
 <add verb="GET" path="CrystalImageHandler.aspx" type="CrystalDecisions.Web.CrystalImageHandler, CrystalDecisions.Web, Version=10.5.3700.0, Culture=neutral, PublicKeyToken=692fbea5521e1304" />
</httpHandlers>

Here are the missing elements that dragging the viewer onto a new page added for me:
Under the assemblies node it added these references:


<assemblies>
 <add assembly="CrystalDecisions.CrystalReports.Engine, Version=10.5.3700.0, Culture=neutral, PublicKeyToken=692fbea5521e1304"/>
 <add assembly="CrystalDecisions.ReportSource, Version=10.5.3700.0, Culture=neutral, PublicKeyToken=692fbea5521e1304"/>
  <add assembly="CrystalDecisions.Shared, Version=10.5.3700.0, Culture=neutral, PublicKeyToken=692fbea5521e1304"/>
  <add assembly="CrystalDecisions.Web, Version=10.5.3700.0, Culture=neutral, PublicKeyToken=692fbea5521e1304"/>
  <add assembly="CrystalDecisions.ReportAppServer.ClientDoc, Version=10.5.3700.0, Culture=neutral, PublicKeyToken=692fbea5521e1304"/>
  <add assembly="CrystalDecisions.Enterprise.Framework, Version=10.5.3700.0, Culture=neutral, PublicKeyToken=692fbea5521e1304"/>
  <add assembly="CrystalDecisions.Enterprise.InfoStore, Version=10.5.3700.0, Culture=neutral, PublicKeyToken=692fbea5521e1304"/>
</assemblies>

Finally it added an item under the handlers node (which I think was the key missing component in my case:


<handlers>
 <add name="CrystalImageHandler.aspx_GET" verb="GET" path="CrystalImageHandler.aspx" type="CrystalDecisions.Web.CrystalImageHandler, CrystalDecisions.Web, Version=10.5.3700.0, Culture=neutral, PublicKeyToken=692fbea5521e1304" preCondition="integratedMode"/>
</handlers>

After updating all that the charts began to show up again!
Hope this helps someone else with this problem, I found there were lots of people reporting the problem and just as many different ways to solve it. This PDF might help you too, it descrbes troubleshooting tips based on the version of Crystal Reports you have, but it didn't end up helping me at all:
Crystal Report Viewers in Visual Studio .NET – Dynamic Images

Installing Ubuntu Linux on HP Pavilion laptop - how to fix screen resolution

2009-11-27T17:50:00.000-08:00

I decided that I would try installing a linux distro on my old HP Pavilion XH485 laptop, since I really wasn't using the system for anything for the last couple of years (it's pretty heavy and windows XP was just running unbearably slow). I also wanted to be able to play with linux again, since the last time I had installed and used linux was years ago when I played with Caldera. I've heard lots lately about Ubuntu linux and it seemed like a logical choice to try on my old hp pavilion laptop, reading a little about the different derivatives (flavors) of ubuntu I looked into kubuntu and xubuntu, ubuntu uses GNOME as the desktop environment while kubuntu uses the KDE desktop environment (which I remember liking quite a bit back in the Caldera days), xubuntu uses a desktop environment called Xfce and uses many GTK+ apps. After reading about the different distros I decided to go with xubuntu 9.10 because many people said that xubuntu runs better on lower performing hardware (even if it is a bit less user friendly than ubuntu or kubuntu), since my hp laptop has only a 1 GHz AMD processor with only 256 MB ram, xubuntu made lots of sense.

Go here for more information about Xubuntu.

The entire installation process was very simple, (first I backed up everything I needed on my existing XP install, see this blog post for some nice hints on how to make that go smoothly) I downloaded the CD image, burned it, went into my laptop's BIOS and made sure the CD-Rom was set in the boot order before the HD and followed the steps in the Xubuntu install. After going through all the wizards I was up and running with my new linux laptop!

The biggest problem after the install was done was that the screen resolution was only 800x600 and I couldn't increase the resolution to 1024x768 and there was about two inches of the screen that wasn't being used in the O/S.

After lots of searching I was able to determine that the system seemed to need a file called xorg.conf in the /etc/X11 directory.

Here is my xorg.conf file:


Section "Device"

  Identifier "Trident Microsystems CyberBlade XP"

  Driver "trident"

  BusID "PCI:1:0:0"

EndSection



Section "Monitor"

  Identifier "Generic Monitor"

  Option "DPMS"

  HorizSync 28-51

  VertRefresh 43-60

EndSection



Section "Screen"

 Identifier "Default Screen"

 Monitor  "Generic Monitor"

 Device  "Trident Microsystems CyberBlade XP"

 SubSection "Display"

     Depth 8

     Modes "1024x768"

 EndSubSection

 SubSection "Display"

     Depth 16

     Modes "1024x768"

 EndSubSection

 SubSection "Display"

     Depth 24

     Modes "1024x768"

 EndSubSection

 SubSection "Display"

     Depth 32

     Modes "1024x768"

 EndSubSection

EndSection

To find out your model of video card as ubuntu detects it, issue the following command:


lspci | grep VGA

Which you should use in the device and identifier lines in the xorg.conf file above where I have "Trident Microsystems CyberBlade XP"

NOTE: If you cannot save the xorg.conf file to the /etc/X11 directory, you may have to issue a sudo chown command to take ownership of the path.

Here are a couple of posts that helped me get this working:

Display problem with Trident Cyberblade Ai1
8.10 Nvidia unable to save xorg.conf.backup

I am now really enjoying my newly reborn old laptop running Xubuntu, after getting this problem of not being able to change the screen resolution figured out I did have some problems finding the right plugings and libraries to be able to play MP3s and DVDs, but using the Synaptic package manager and google it wasn't too difficult to find out what patches I needed to apply. Now I have a pretty sweet media player machine that would have gone to the electronic recycling otherwise!

Associate SqlServer user with login after backup restore

2009-11-19T15:53:00.000-08:00

After backing up a SqlServer instance and restoring it to a new database or machine, you will find that the security is all messed up (logins are no longer associated with the users). If you go in through the Management Studio, there is no way to fix the association either, but luckily you can run the following command to update the user with the correct login and fix the credentials:


alter user [user_name] with login=[login_name]

NOTE: if you can't get this command to work on your SqlServer 2005 instance, make sure you have the latest service packs installed, I think they added the with login portion of the alter user command SP2.

Backing up and Reformatting Windows XP Computer

2009-10-12T00:35:00.000-07:00

This is always such a hassle, your computer is at it's life's end, you've had the system for over 3 years and you've installed and uninstalled programs, games, and who-knows-what else over the past long while and it's time to backup and reformat to squeeze the last bits of performance out of that bad-boy comp you bought 4 years ago. You've upgraded the memory and video card so it's not so out of date and perhaps malicious infections, viruses, and/or spyware cause your computer to not live up to it's it's potential.

Time to backup your files to an external source and reinitialize XP or go to Windows 7. Here are some of the directories in XP you'll want to look for important files to backup:

C:\Documents and Settings\[UserName]\Desktop
C:\Documents and Settings\[UserName]\My Documents
C:\Program Files

As well as any files you might have saved directly under the C:\ (root) of your hard drive or subfolders in the root. Note: there are also hidden folders inside "Documents and Settings" such as local settings and application data that might contain saved games or other important files you'll want to backup.

To make sure I got all my files under C:\Documents and Settings while backing up I used robocopy to copy files off onto a separate hard disk I connected using this cool SATA/IDE to USB adapter I bought awhile back and my 500 gig Seagate HD. Here is the command I used to copy the files:

ROBOCOPY "C:\Documents and Settings" E:\Backup /COPYALL /B /MIR /R:0 /W:0 /NFL /NDL

This took a really long time, especially because I had some photos on my desktop that I hadn't moved before the backup, but at least I could have the peace of mind knowing all my files were transferred despite the failure of copying through windows explorer giving the file in use error.

You could also run Robocopy for "C:\Program Files" but I found it was easier just to do a spot-check on those folders since most modern programs save their user profile and customizable data files inside the user's Documents and Settings directory.

Before you make the plunge of reformatting to avoid the headache and pain of lost data, use disk analysis tools such as WinDirStat to make sure all the data you need to backup is archived off your main drive. If you need to make backups within installed programs such as Local Mail Systems (i.e. Thunderbird and Outlook), Databases (such as MSSql, MySql, Oracle, etc...) or Source Control Systems (i.e. CVS, SVN, VSS, etc...), perform those backups with the appropriate programs and make sure you have the files archived somewhere safe.

Once you're ready, go for it! Make sure you have your XP or Windows 7 disk and hardware driver disks handy as well as files for other programs you use regularly. If your computer boots up at the BIOS screen but does not prompt you to boot off the CD ROM for the O/S you may have to go into the CMOS screen to change the boot order to include the CD/DVD before the HD.

It is so nice to run a clean system right after a reformat, of course you'll want to install your Chipset drivers (which includes sound drivers if the motherboard supports it) as well as your video drivers and reinstall an anti-virus solution as soon as possible, I suggest AVG Free, it is a very functional virus scanner for only $0!!!

If everything goes smoothly you'll have a working system that runs much more efficiently than before, I'm still looking for a good anti-spyware program and had been using AdAware before but if anyone has found a better free spyware-killer let me know.

Update October 13th 2009:

I've decided to try Microsoft Security Essentials which serves as both my virus scanner and my anti malware and spyware solution, and it even works in XP!

To be fair, I hadn't performed a Full Scan within AVG before I uninstalled it on my fresh XP install but Microsoft Security Essentials says "Preliminary scan results show that malicious or potentially unwanted software might exist on your system." during it's full scan, so I patiently wait for it to finish so I can see what it has detected...

It found one: "Spyware:Win32/Aureate" item and also: "Backdoor:Win32/Trenk!rts" which it was able to clean out. Overall I'm pretty happy with the Microsoft Security Essentials application, the UI is nice and clean and since it is free and provides antivirus, and spyware/malware protection I know I'm pretty well covered.

Windows Time Service: W32Time - Synchronising time between servers

2009-09-22T11:29:00.000-07:00

We have the need at work to Synchronize the clocks of our application servers with the database server which should be a relatively simple task and it's not too difficult given you have a domain controller in your network, however we don't want to require a domain controller for our software to run properly and getting the Windows Time Service to sync the clocks between servers without a Domain controller was a bit of a headache.

In this article: How to configure an authoritative time server in Windows Server Microsoft describes the different registry entries used in Windows Server 2003 to configure the W32Time service, if you also read the article: Configure a manual time source for a selected client computer you will become more familiar with the commands for querying and synchronizing against a remote time server.

First I tried following the directions under: "Configuring the Windows Time service to use an internal hardware clock" because I didn't need the server to sync to an external time source, I just wanted to sync the clocks of the client machines to my database machine.

After configuring the server I issue the following command from the client machine:
w32tm /config /manualpeerlist:peers /syncfromflags:manual /update

Where peers is the server's name.

However as soon as I issued the command:

w32tm /stripchart /computer:target /samples:n /dataonly

from the client machine, the command errored out with: error: 0x800705B4
Or if I give the resync order to the client machine using the command:
w32tm /resync
it replies back with:
"The computer did not resync because no time data was available."

After searching on Google and not turning up anything very helpful I went ahead and followed the instructions under the "Configuring Windows Time service to use an external time source" section. For the registry entry that dictates which server to use as the external time source (HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters), I used the value: time.windows.com,0x1

Using this setup I was able to get my database server to sync with time.windows.com and my web servers to sync with the database server (however at one point I had my db server synced up with the Domain Controller (DC) and it was working that way as well so I'm not sure if performing that step made it work even after changing the configuration to externally sync up with time.windows.com). As I'm writing this blog post I'm retracing my steps on another set of machines and I can't get it to work so maybe there is some sort of problem with having IIS on your time server because I found other posts related to that.

Concluding, time syncing in windows is a real pain (to say the least) and it seems that getting it to work in an environment without a Domain Controller is even more difficult. I'll update this post as I find new things or if we need to setup another cluster I may learn more. Hopefully this helps someone, if you have more experience with this and know what the magical combination of steps and registry entries are to make it work let me know I'd love to hear. I'm afraid that if I have to figure this out again on another cluster I'm going to be in real trouble.

Update: Feb. 23rd 2011 -
It looks like the problem I hit when trying to set this up before was a conflict with the automatic sync setup with the windows time server time.microsoft.com, if you right click on the date in the task bar and click the Internet Time tab, make sure you uncheck the checkbox: "Automatically synchronize with an Internet time server"

Then once this has been unchecked on your client machines, follow the instructions under: the "Configuring Windows Time service to use an external time source" section of the following article: How to configure an authoritative time server in Windows Server in order to setup the server machine.

Next issue this command on the client machine:
w32tm /config /manualpeerlist:peers /syncfromflags:manual /update

And then to make sure your client's clock is close to the servers, issue this command:
w32tm /stripchart /computer:target /samples:n /dataonly

Finally you'll want to setup a scheduled task to run your resync command on the client every hour or so (depending on how accurate your servers need to be with each-other):
w32tm /resync

DupeMan - Duplicate File Management Program Updated

2009-04-02T23:09:00.000-07:00

It's been about 3 years since I've updated and released changes to my original DupeMan Duplicate file management tool, but I've been able to make changes to it in my spare time and when I get bored, I upgraded to .NET 2.0 framework and added the ability to save and load the results of a check for duplicates indexing process as well as a new MP3 Playlist dockable control.

Download DupeMan Here.

Feel free to pass this program or link on to your friends or anyone you think might be able to use it (those who's collection is more quantity than quality).

If you do install it and try it out, let me know any feedback you have,
-Matt Palmerlee

(Note: you need to have at least .NET 2.0 installed on your system to work.)

Configuring Google Apps to host your Domain for free

2008-06-04T18:10:00.000-07:00

I've recently setup a new domain name for a new S/W Development Consulting company I'm creating called Mastered Software (MasteredSoftware.com). I was planning on creating my new website using C# .NET and Visual Studio .NET 2005, because that is the platform I'm most comfortable with, however, finding affordable Windows hosting that supports the .NET framework is not easy, the cheapest I could find was $10 a month if you bought 2 years at a time, which isn't much compared to other hosting solutions, but for a small business even this adds up quickly.

I was also recently checking out Google's App Engine and Google Apps for your domain services, and what's cool is that you can host your domain and even email through the google apps for your domain service! The best part is, for small companies that don't need lots of storage and features it's free!

Here are the steps I took to have my domain hosted by google:

Start by signing up for Google Apps for your Domain at: http://www.google.com/a/
You will be prompted to verify ownership of your domain during setup, there are a couple of options to do the verification, including adding a special CNAME record in your domain name registrar's system to point to google.com, this is the method I used for verification because it was the most straightforward. While you are there, you may want to add additional CNAME records for your domain which point to ghs.google.com (i.e. masteredsoftware.com and www.masteredsoftware.com).
Enable the applications you want such as email, calendar, etc... For creating the content for my main site I choose to use the Google Pages application because it's very easy, configure the web address for google pages to point to www. for my site it is www.masteredsoftware.com click on the "Change URL" link from the Web Page settings screen to update the web address. Create a basic home page for your site through google pages so you have something to look at when you go to your domain, don't forget to publish your changes so they are live on the site.
At this point you should be able to browse to the "www" version of your site (if you setup the CNAME records in step2 for www.yourdomain.com to point to ghs.google.com).
Unfortunately there is no way at the moment to have the "naked" domain version of your site point to your google pages site ,unless perhaps you have your domain hosted through google domain hosting, I don't know because I haven't tried that service. You maybe able to setup a forward or redirection from the naked domain to the www version in your hosting provider's DNS settings, with my hosting provider I was not able to do that and have some more of the advanced features for my domain such as redirecting the MX records to google. Fortunately you can host your naked domain using the google apps framework.
Once you have signed up for the google apps framework, add your naked domain (in my case masteredsoftware.com) under the google apps framework options.
Follow the instructions on this page: Uploading your application to upload your files to the google app engine service.
Here is the code for my masteredsoftware.py file:
print "Status: 302 Moved"
print "Location: http://www.masteredsoftware.com/"
print ""
Also setup your app.yaml file pointing to the main script file:
application: masteredsoftware
version: 1
runtime: python
api_version: 1

handlers:
- url: /.*
script: masteredsoftware.py
If everything worked out the code files above should forward your naked domain requests to your www.yourdomain.com version.
...Finally you might want to add google analytics to your pages site so that you can track visitors. Just click on the area you want your analytics code, click the edit html link (on the bottom right of the google pages page), surround the analytics code with div tags and save and publish your changes.
Now you can play with some of the other cool features that Google apps for your domain allows (such as gmail hosting, etc...)
Have Fun domain hosting!

VSS to SVN Migration

2007-11-14T13:45:00.000-08:00

The last couple of weeks at work I've been moving our Source Control tool from Visual Source Safe (VSS) to Subversion (SVN). There are many reasons to move away from VSS, mainly it just sucks, random corruption, that nice error message about having to run the analyze tool, and just the fact that it is a very poor Microsoft Product. Also VSS doesn't support some of the features we want as we grow such as branching and merging.

There were other Source Control options we investigated when looking to migrate off of VSS, the most logical was to move to Microsoft's Team Foundation Server (TFS), which I've heard is very robust and is an easy switch from VSS, it also apparently has some really nice features for controling Database Schema and Code (Stored Procedure) changes. The problem with TFS was, to work effectively (in addition to Costly TFS licensing) our Visual Studio Developers would have to upgrade to the Visual Studio Team System which is an additional cost. Our decision was to go open source, SVN is a very popular technology for Source Control and it supports many of our desired processes for Source Change Management, such as branching and merging releases.

Here are some helpful links for SVN and the conversion from VSS:
Good Overview of the Upgrade Process

Subversion Main Site

Tortoise SVN Client

VSS to SVN Migration Tool

TamTam SVN Visual Studio SCC Provider Plugin

Since we wanted to retain the check-in history from Visual Source Safe, we used the VSS to SVN Migration tool listed above. I made some modifications to the tool as well, I wanted to keep the user who originally made the change as well as the date the change was made (the 2nd version of the Migration tool just uses one sign-on to SVN and just uses the current computer time for the check-in time).

My change parses the user name out of the Version information from VSS and uses this login instead for SVN, it also parses the date and time the checkin was made, switches the computers clock to that date and time, and then finally does the commit to the SVN repository (which is on my local machine anyways because I couldn't get the the program to work on Windows 2003). I did run into some problems where on some files I couldn't get the versions from VSS for some reason, maybe these files were from a backup/restore in VSS and that is why it had trouble, I'm not sure but I was able to manually import the files that had this problem and the rest of the process completed smoothly.

Here is the direct link to my Modified Version of PowerAdmin's VSS to SVN Migration tool (I sent it to them, but in case they don't repost my version, here it is):
VSSMigrate3_code.zip

The next step was to move my local SVN repository to the final destination server, which can easily be done with the Subversion dump and load commands as described on this page:
How to move a Subversion Repository

Finally, I've setup backup scripts on the Subversion Windows 2003 Server, we had a similar set of backup batch file scripts running to backup our VSS database and FTP the backup file offsite. These batch files were then setup to be run nightly and weekly via the windows scheduler. The new Subversion backup batch files perform a svnadmin hotcopy to make a copy of the repository without taking it offline, then it zip compresses the files from the hotcopy using the 7zip program's command line arguments, it finally creates a remote ftp script that is executed to ftp the compressed backupfiles offsite. Here are the contents of my fancy batch file backup script for Subversion:


FOR /F "tokens=1 delims= " %%i IN ('date /t') DO SET DATE=%%i

cd C:\Program Files\Subversion\bin\

RMDIR /S /Q D:\backup\SVN\current\
MKDIR D:\backup\SVN\current\

DEL D:\backup\SVN\%DATE%_SVN_backup.zip

svnadmin hotcopy D:\svn_repo D:\backup\SVN\current --clean-logs

"C:\Program Files\7-zip\7z" a -tzip -r -y d:\backup\SVN\%DATE%_SVN_backup.zip d:\backup\SVN\current

cd D:\backup\SVN\

> script.ftp ECHO username
>>script.ftp ECHO password
>>script.ftp ECHO cd ~/backup/SVN
>>script.ftp ECHO binary
>>script.ftp ECHO prompt n
>>script.ftp ECHO mput %DATE%_SVN_backup.zip
>>script.ftp ECHO quit
:: Use the temporary script for unattended FTP
FTP -v -s:script.ftp ftp.servername.com

After that developers must rebind their Solutions to SVN, which can be quite a pain, but once it's setup using TamTam, going forward it is smooth opening the solution files.
One tool that helped me out upgrade the solutions (especially VS2003) was Clean Sources Plus which removes source control bindings from the windows explorer right click menu.

ASP.NET Web Application Project Upgrade - Assembly Reference not found

2007-11-14T12:49:00.000-08:00

We've recently undergone a conversion to Visual Studio 2005 Service Pack 1 (SP1) for the new Web Application Projects (WAP) they have added.

The Web Application Projects give more control over the files included in a Web Site because now you have a project file similar to the VS 2003 Web Application model.

There are a couple of nice guides online to upgrading a Web Site to a Web Application Project after installing SP1, here is one I followed for the Upgrade:
Scott Gu's Migrating a Web Site to Web Application Project.

Basically, you just create a new web application project, copy in all your pages and other files, then on all the ASPX pages, right click and choose "Convert to Web Application", which creates the filename.designer.cs files necessary for the new Web Application Model.

To simplify the upgrade initially I didn't add namespaces for the separate App_Code files and I just put them in a folder named "Classes".

After that there weren't many more hoops to jump through for the site to successfully compile.

One reason for the upgrade is to make it easier to create a Web Setup (Installer) for the Web Application Projects where making a Setup Project for the original "Web Site" project was a pain.

After creating the Web Setup Project for our Web Application Project, then testing the installer, I kept getting an error:
"The name 'Namespace' does not exist in the current context"
When I viewed the detailed compiler output: "The type or namespace name 'Namespace' could not be found (are you missing a using directive or an assembly reference?)". This was because the dll files for the website were not in the bin folder underneath the web application root folder.

Once I fixed this in the installer (I manually added the bin folder available from the "View -> File System" context menu, and added outputs: Primary Output for each web project and library as necessary).

Overall, I'm much happier with the new (or old) Web Application Projects model in Visual Studio 2005, it gives the developer much more control over the behavior of the website, the included files, simplifies deployment, and is much faster to compile and test.

Read Foreign Region DVD

2007-11-02T22:34:00.000-07:00

I recently got back from a 3 week vacation to Spain and Italy, while I was over there we ran out of space on our camera xD drives so we went to an Internet Cafe in Italy and had them burn a DVD of our pictures that were on our 2 GB xD card. After that I nervously formatted that card so that we could continue taking pictures of our trip.

Once I got back to the States and started compiling all our photos (over 2000 of them!) I insert my DVD with my photos into my computer's DVD drive and I hear it clicking and thrashing and my heart drops into my stomach.

When my DVD ROM couldn't read the disk I paniked, I tried to keep my calm; telling myself it was alright, the files were there, I just had to get them off it.

After much searching online for a solution, I realized the problem was my DVD-ROM was the 2nd generation computer DVD Player with this Movie Industry Region protection so the movie industry can control releases of movies within different countries. What a crock!!!

Luckily I was able to find a firmware upgrade tool that allows my model of DVD Rom to be essentially Region-Less (or region 0) so the DVD player doesn't care what region the DVD is actually encoded for.

After I upgraded the Firmware and used this DVD Region Killer s/w to remove the region protection from windows I could finally read my dvd and use windows to get my photos off of it.

Note: do not leave a foreign region DVD in the drive on bootup or you are bound to get hangs as the O/S looks at the drive to boot from.

HttpWebRequest in C# for Web Traffic Simulation, Watch out for Expect 100 Continue and Nagle

2007-09-04T16:44:00.000-07:00

Over the past few months we've been busy at work creating our scalable Intelligent Chat application using C# .NET 2.0 and AJAX. To perform load testing against our application we have written a simple Traffic Generating/Load Simulating Windows Forms application which makes Asynchronous HTTP Requests using the HttpWebRequest object, it is pretty straightforward and has been able to surface many problems in our main application. We did come across a big "Gotcha" in our simulator, after looking at the generated network traffic we noticed that in the header of each request the simulator was sending "Expect: 100 - Continue", after which the server would send the 100 - Continue response to the client. Since this was not the correct behavior, we added the following line of code:
System.Net.ServicePointManager.Expect100Continue = false;
As indicated in the following blog posts:
HttpWebRequest and the Expect: 100-continue Header Problem
HttpWebRequest and Expect 100 Continue

After this change however, the headaches started, everything was fine when we tested in our Development environment, IIS on Windows 2003, when we deployed the simulator into the Test environment (as far as I can tell, exactly the same server configuration, etc...), the simulator was so slow that requests would time out almost immediately.

After using Wireshark to view the network traffic in the test environment, we were seeing some strange behavior, it looked like the simulator was sending the header of the POST request, and then waiting for the Acknowledge (ACK) message back (which would take more than 200 milliseconds) from the server before sending the body of the post. Thanks to our brilliant network engineer we were able to determine that the client caused this behavior by using the Nagle Algorithm for it's requests.

That is when we put in one more line:

System.Net.ServicePointManager.UseNagleAlgorithm = false;
In our simulator and it seemed to fix the problem in the test environment, I have no Idea what is different in our development environment, some IIS configuration, maybe the fact that our test environment servers have .NET 3.0 installed? It is still a mystery to me. I figured I would share our experiences with the Expect 100 continue and Nagle Algorithm to perhaps save others some of the same pain we dealt with debugging this issue.

Update September 7th, 2007:
We found the difference in our development environment, the windows 2003 server in our development environment is R2 with Service Pack 1 (sp1) and the windows 2003 server in the test lab is win 2003 R2 with Service Pack 2 (sp2), after viewing further network traces this seems to stem from IIS sending a 100 continue under sp1 (even when we don't send the expect 100 continue message), then what we see is the client waits a long amount of time before sending the ACK message back to the server (I suspect because it has already sent the entire body in the initial POST headers).

This Blog post points to a hotfix that addresses this issue in sp1:
HTTP.SYS, IIS, and the 100 continue

I've put together a simple table to show all the combinations related to 100 - Continue and Nagle, and the results of the test (it was just too hard to keep it all straight in my head):

Results	Server Sends 100 - Continue (Windows 2003 SP1 && XP IIS 5.1)	Client sends Expect: 100 - Continue	Use Nagle Algorythm
Fast (extra traffic sent)	1	1	1
Fast (extra traffic sent)	1	1	0
Fast (didn't send POST body, after the 100 - Contine it has something to send?)	1	0	1
Slow (waits before sending ACK to 100 - Continue)	1	0	0
Fast Responses	0	1	1
Fast (extra traffic sent)	0	1	0
Slow (Nagle causes a ~200ms wait for ACK)	0	0	1
Fast Responses	0	0	0

(Note: A zero in column one means Windows 2003 SP2)

SQL Server Outer Join with where clause bug

2007-06-15T14:57:00.000-07:00

I spent most of the day trying to figure out one simple outer join query in SQLServer 2005, it turns out that the problem stems from the WHERE clause, apparently SQL Server will execute the WHERE clause before the join? Which doesn't make much sense to me, it seems like a bug, but whatever, at least I only wasted about six hours trying to figure out that once you put the criteria within the join clause it works as expected.

Here are some more articles I found on the subject:
Outer Join trouble
Additional Criteria in the JOIN Clause

C# DrawImage and overriding the OnPaint method

2006-07-30T18:33:00.000-07:00

I recently spent hours looking for a solution to a strange problem I had in my custom user-drawn control when using the DrawImage method, and I thought I'd share my misery and hopefully keep you from wasting your time figuring this out as well. What was happening was that using images that were not 96 dpi were scaled wrong using the DrawImage method in C# (I also tried using the DrawImageUnscaled methods to no avail). I figured that the DrawImage method should account for the different screen resolutions and I wouln't have too, but it turns out that there is a SetResolution method on the Image (Bitmap object). So I delved further on how to get the system's resolution (or dots per inch, dpi) from .NET and I ran across this page:
How to: Handle Orientation and Resolution Changes that talks about the Microsoft .NET Compact framework, but there was a very interesting tidbit near the bottom of the page:

"If your application contains graphics that are drawn in the OnPaint method, they will not scale automatically. You will need to use the DpiX and DpiY properties of your Graphics objects to determine appropriate scaling."

That was all I needed, one line in my OnPaint method and my images are now scaled properly:
(g is my Graphics object)

this.myBitmap.SetResolution(g.DpiX, g.DpiY);

It's amazing how long it takes to find the answer to questions when you don't know how to ask google for them!

Moving Local User Profiles to Domain and MSI Install/Uninstall Nightmares

2006-07-24T11:39:00.000-07:00

We just got through moving most of our Windows XP computers here at work onto a new MS Exchange Server with Active Directory, Domain, etc...
One major headache was actually moving the user profiles over from a local user to a domain user.
Since you can't just change an existing local user to a domain user, you have to go through major hoops and cross your fingers everything will work with your new user (or just do a complete reinstall/reformat of windows which is not all that fun).
The first thing we did was move the computer onto the domain by using the System window available from the control panel or by rightclicking My Computer and clicking properties.
On the computer name tab, click "Change" and change from "Member of" workgroup to your new domain server.
Once you've switched over to the Domain, next comes the tricky part, actually making the new user, ensuring correct permissions are set, and copying the profile from the old local profile to the new domain profile.
To create the new Domain profile on your machine, all you need to do is log off from your local user and login to your Domain account using your username and password that is on the Domain Controller and selecting the domain from the login prompt.
After you've sucessfully logged in as your new domain user, log back out and login as the Administrator account for your system. Right click on My computer and click the Manage button to bring up the Computer Management window, click on Local Users and Groups, then the Groups subfolder, then right click on the Administrators group and you should make sure that both the old local user and the new domain users are in the Administrator group. Also ensure that the folders under C:\Documents and Settings\ and C:\Documents and Settings\ have full control for administrators (you may have to disable simple file sharing to modify these security settings in XP by clicking tools -> Folder options -> View Tab, then uncheck the last checkbox in the list labeled Advanced.
Once you've gotten all the security/permissions setup it's time to actually do the copy, I would recommend a full shutdown and reboot because you never know what files are still in use with windows, then sign in again as the Administrator once the system is back up.
Right click My Computer and click Properties to bring up the System window once again, click the Advanced tab, and then click the Settings button in the User Profile section to bring up the User Profile window, select the local profile you wish to copy from, click the Copy To button, and in the popup window select the new Domain user as the "Copy Profile to" profile, and I would also make sure the "Permitted to use" is set to Everyone.
If everything goes ok it will take a few minutes (depending on the size of the local profile) and your domain user will now have all the settings, desktop, registry entries, etc... that your old local profile had.
Sometimes this fails, especially if there are files in long pathnames under the C:\Documents and Settings\ directory, if it fails you have a couple of options, either remove those nasty long path files it errors on, or you can try a manual copy in windows explorer instead of using the "Copy Profile to" tools, but who knows what other registry settings, etc... you'd be missing if you do a manual copy in windows explorer so I'd recommend trying to use the Copy Profile tool if possible.

Even after this, the struggles with installed programs are not over, if you try to run a program that was installed just for the old local user instead of installed for all users with your new Domain User, you'll find the program will probably not run and windows will complain looking for the originally installed MSI file, which is a complete pain because who keeps the original MSI/Setup files they installed programs with? You'll also find trying to uninstall these programs will also fail through the Add/Remove Program files tool for the same reason. After lots of searching in the windows registry and trying to find the registry key that identifies the actual version installed so I could get the appropriate MSI file to correctly uninstall the corrupt program, I found that the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall key holds some interesting values such as UninstallString, however I couldn't figure out how that hash code related to the "Revision Number" in the Advanced Properties for the MSI file, or any other attribute. Finally, giving up, we found that if you try to uninstall from Add/Remove Programs, to remove it from the list (even if it fails) then reinstalling the program but selecting "All users" instead of "Just me" in the install options made the install go through sucessfully.

DupeMan - Duplicate File Management Program

2006-02-13T16:21:00.000-08:00

Over the past year or so I've been working off and on in my spare time on a program to easily find duplicate files on your system. This project was born for the desire to easily find duplicate MP3 files. I have a large collection and so do my friends, but nobody wants duplicate songs of the onces they already have. I've recently been polishing up the system and it's now not completely embarrassing, so I've released a Beta version. It works really well with MP3 files, it reads ID3 Tags to find similar files and notifies you of duplicates, it will also find untagged or "blank" ID3 tags and allows you to quickly and easily update the tags (based on folder and file name).

Download DupeMan Here.

Feel free to pass this program or link on to your friends or anyone you think might be able to use it (those who's collection is more quantity than quality).

If you do install it and try it out, let me know any feedback you have,
-Matt Palmerlee

(Note: you need to have at least .NET 1.1 installed on your system to work.)

Multi-Threaded SqlServer and .NET Database Connections

2005-04-05T22:02:00.000-07:00

Using MS SqlServer and .NET in a Muti-User or Multi-threaded environment doesn't behave well when using one Database connection object for all your queries. After the last couple days of database nightmares I've found it's always safer to just instantiate a new db connection object each time you want to perform an ExecuteQuery, ExectueScalar or ExecuteNonQuery. The object of only having one database connection is to reduce the overhead of multiple connections and increase performance, and maybe there is a way to keep a persistant connection to the db that manages concurrent inserts/updates/delets/selects, but in the sake of time and easy of programming just open another connection! Also remember to close all your connections after you use them.

.NET Web Services and SOAP

2004-07-28T09:34:00.000-07:00

A couple of weeks ago, I spent the better part of a day banging my head against the wall when trying to add a web service resource to a project in Visual Studio .NET 2003. In my test project, the web reference worked fine, but as soon as I tried to add it to my production project, it couldn't find the type I was trying to create, even though the web reference was there, as if the namespace was different for the web reference than the rest of the project. There is no namespace property for the web reference, but it takes the value for the "Default Namespace" property on the project.
The problem for me was I had changed the namespace for all my source files, but never updated the default namespace property on the project, so the web service reference was being added under a different namespace than my source C# file that was referencing it.

Moral of the story, if you change the namespace for the source files in your project, make sure you right click on the project, then properties, then change the "Default Namespace" property to the new namespace you want.