jjstautt

So we just learned a good, hard lesson with Sun Microsystems. As I have mentioned before, Perimeter was hitting quite the annoying bug with the 7410 that would cause Microsoft Office documents to take up to 45 seconds to load, save, autosave, etc. This issue was big enough that mid-April we had to assure our COO that someone (uhh, Sun’s employees are someone) was working around the clock until it was fixed. When we reported the issue with Sun we were assured by their support team that the next quarterly release would include a fix.

On April 27, the 2009 Q2 update was released. I diligently scheduled an upgrade as soon as possible. I installed the upgrade on Saturday May 2 and all seemed fine. When I arrive to work on Monday, I get some complaints about Microsoft Outlook and Adobe InDesign randomly crashing. After some investigation, the issue involved open files on the CIFS share from the Sun Storage. In the error log in all cases were “delayed write failed” messages. More investigation showed that CIFS kept restarting itself. Well, if the CIFS sessions are ending with a CIFS service crash, that would explain the delayed write failed issues.

So again, off to contact Sun technical support. They walk me through submitting a support bundle and let me know they will get back to me. Ok, fine. Until May 5 rolls around and the Storage system panicks in the middle of the day. Yay, more CIFS bugs: “reboot after panic: mutex_enter: bad mutex”.

Support finally gets back to me the next day. “There are so many bugs related to CIFS in this release that it is impossible to determine which ones you are hitting. The Q2 release is an entire new release of Solaris, so it’s not possible to backport any of them to the previous release. The developers are working on a fix. Since it’s currently in development we can’t predict a release date.” After this I had to work quite hard on trying to get a recommendation on what to do out of the support technician. Finally, he recommended that we revert to the latest 2008 Q4 release.

Ok! Let’s revert… but, wait. That would have to be scheduled. So I ask management if I can revert that night after hours so that we don’t chance losing any data. I’m told I’m not allowed to and that the Microsoft Office issue was big enough that unless the system panicks again I’m not to do anything to it. Risk of losing data is less important than having to wait a few seconds for a file to open.

Fast forward to May 12. Luckily no more system panicks yet. Sun releases patch 1 to their 2009 Q2 release. I schedule the install for the next evening. We have now been running with the 2009.Q2.1 release since May 13 without any issues. Yay!

So, my point and hard-earned experience: don’t upgrade to the first revision of a quarterly release from Sun!