At long last, Microsoft today released a number of communications around the long-standing Windows Home Server Data Corruption bug, providing more details on the cause of the bug and outlining plans to release a fix for the issue in June 2008.
In a post today on the Windows Home Server Team blog, the team stated that Microsoft now understood the bug “really well”, and that whilst a fix for the bug had been partly coded, we should not expect to see a final fix released until June at the earliest.
Microsoft also updated the Knowledge Base article covering the issue today, with more detailed information on the bug, and reiterating their belief that “most users are unlikely to be affected”. However, the article now states that the issue can occur when files are transferred as well as edited and saved within identified applications.
As many have suspected, the bug is related to Windows Home Server’s Drive Extender technology, which is one of the major areas of innovation within Windows Home Server, managing file transfer across multiple drives, and negating the need for drive letters. Specifically, the bug occurs within WHS’ file system mini-filter driver, as stated in the KB article:
Windows Home Server uses a file system mini-filter driver in addition to the NTFS file system to implement Shared Folders storage technology. File system mini-filter drivers are an extensibility mechanism provided by Windows to enable storage scenarios. For distributing data across the different hard drives managed by Windows Home Server, the Windows Home Server mini-filter driver redirects I/O between files that are stored on the main hard drive and files that are stored on other hard drives. This redirection mechanism is only enabled when Windows Home Server is managing the Shared Folder storage of multiple physical hard drives. A bug has been discovered in the redirection mechanism which, in certain cases, depending on application use patterns, timing, and load, may cause interactions between NTFS, the Memory Manager, and the Cache Manager to get out of sync causing corrupted data being written to files.
Given the bug hits at such a low level of the operating system, fixing the bug successfully will require a major piece of re-engineering within Windows Home Server, hence the delay in publishing the fix. The fix will be internally tested at Microsoft, before release to (my assumption is a limited number of) external participants in the coming months - this will ensure that the bug works correctly and does not break other areas of the operating system. The fact that MS are actually beta testing the fix is a signal of the deep level engineering required to fix the bug, as well as the intent to get the fix right upon release.
The KB article goes on to advise users to use Windows Explorer or a command line tool to copy files to and from the Windows Home Server as well as to set the Shared Folders on WHS to Read Only. Importantly, the article also advises against directing applications (they didn’t name them, but I will - we’re talking iTunes and Windows Media Player here) to access files on your shared folders which may attempt change metadata without user input.
Microsoft have requested those “with evidence” of (i.e. a reproducable method of incurring) the data corruption bug to get in touch directly at whsforum@microsoft.com.
Whilst there’s not a lot of good news in today’s announcement, I’m much happier that Microsoft are now communicating more directly with the community, after a period of silence - I’d encourage them to keep the channels open and build on the great work they did in establishing and working with the community during the WHS beta over the coming months.
More Info: WHS Team Blog | MS Knowledge Base
March 10th, 2008 at 6:59 pm
Microsoft improved their communication policy. Nice.
Is it something protecting my data from loss? Nope.
What if I got mycar to my mechanic with defect brakes? Would I be happier if he didn’t answer the phone for weeks and then finally said, he understood the problem? Where is the repair?
March 10th, 2008 at 7:04 pm
Most users will not be affected? I beg to differ! I am sorry, as my as I was excited about WHS, it seems that Microsoft brass managed to mess it up … again. It is evident, that when released WHS was not even a solid beta product. How can anyone at Microsoft feel comfortable with a situation where a backup , data sharing product does not perform its basic functionality (the one that was being sold)? Do they grasp the concept of product “recall” in USA? And what about their patterns like HP? They put them in a VERY unpleasant situation with a lot of possible product liabilities. No wonder that MSFT stock has done virtually nothing in last decade and a half. Every time the “gurus” from Microsoft do something, they provide new reason to pay billions to US and EU governments in penalties. I am almost convinced that by now the EU budgeting committee has a income line item - “Microsoft screw-ups income”. What a joke!
Do we know if somebody was fired for this (QA, product management, marketing …)? I hope that at least a dosen of them was sent to “pursue some other busines opportunities”.
March 10th, 2008 at 7:11 pm
I wrote this message on microsoft forum.. it’s been deleted with another user’s post complaining about the KB update. What a shame!
I’ve just bought an OEM copy of french WHS (after spending a month testing the 120 days trial) hoping that this Monday KB update will offer us a fix.. June 2008?! Come on! you’re kidding me!
I’m really upset, i’ll have to wait three months (at least) using a single HDD without duplicating my data. Great! I have to deal with 500Go of very precious (i mean emotional value) non duplicated family media!
WHS = Bakup your backups.. maybe using a “real” server solution (how about linux?)
I’m really thinking about returning my WHS dvd without even opening it and contact a consumer rights defense association.
March 10th, 2008 at 7:15 pm
Reivised version:
Most users will not be affected? I beg to differ! I am sorry, as much as I was excited about WHS, it seems that Microsoft brass managed to mess it up … again. It is evident, that WHS, when released, was not even a solid beta product. How can anyone at Microsoft feel comfortable with a situation where a backup , data sharing product does not perform its basic functionality (the one that was being sold)? Do they grasp the concept of product “recall” in USA? And, what about their partners like HP? They were put in a VERY unpleasant situation with a lot of possible product liabilities. No wonder that MSFT stock has done virtually nothing in last decade and a half. Every time some “gurus” from Microsoft release a product, they provide a new reason to pay billions to US and EU governments in penalties. I am almost convinced that by now the EU budgeting committee has a income line item - “Microsoft screw-ups income”. What a joke!
Do we know if somebody was fired for this (QA, product management, marketing …)? I hope that at least a dosen of them was sent to “pursue some other busines opportunities”. This fiasco proves that the quality control / testing procedures do not follow industry best practicies procedures. Why would anybody want to hire Microsoft Consulting Groups for their project, where this company has basic problems in delivering their own?
March 10th, 2008 at 7:29 pm
The line “most users will not be affected”…continues to drive me crazy. WHS is still an early adopter product. Those kind of people are going to use the fileshare functionality. They are going to use application that change data on a shared file. I would bet the more like 70-80% of WHS users are affected by this. Many may not know it.
Just put a onenote shared notebook out there…and within 5 minutes you can corrupt the notebook. Not hard at all.
Use command line to copy files. Mark a fileshare as read-only??? Are you kidding?
I continue to love the idea of WHS and my EX470…..but it is going to take a while for me to trust it after it is fixed.
March 10th, 2008 at 7:52 pm
I agree with many others. This is absurd…. June 08!!! That puts the total fix time at 6-7 months! What this really means is that they are selling and advertising a non-functioning product for at least another 3 months and total of around 6-7 months.
I like the idea and the trial version shows its a great concept. I purchased the ex470 but have not opened and will be returning. I guess the idea of waiting until the first SP/Fix holds true for MS products.
I intend on evaluating a linux solution more closely. It is going to take some serious effort by MS/WHS to win this customer back.
At least they are attempting to improve communication but even it could still use some work.
March 10th, 2008 at 8:36 pm
Ok…so we can start bashing on MS (and they really, really deserve it)….but I still the owner of a self build homerserver with 2TB of (duplicated and original) MP’3’s, photo’s and DVD’s… I think (no I’m not a microsoftie) we should make the best of a bad situation….and what we really need is good advice (hint, hint to the we-got-served moderators) to avoid loss of data….
I have a few questions …. and hope someone will answer them :
- If I make all my shares read-only (as suggested by MS), can WMP11 still make changes to my collection of MP3’s ??? And if so, how can I avoid this.
- If I make all my shares read-only…how can I still copy files to it ?
- “..depending on application use patterns, timing, and load,….” (from the KB), well I use my WHS mostly as a read-only fileserver (I rip my dvd’s, cd’s locally and then copy them to the WHS)…Can I estimate the change of me being affected.
- Lastly : is there a way to check wether the files are corrupted ??…
Greetz
Geert D.
March 10th, 2008 at 8:52 pm
Hey Geert
The best advice I can give is to follow the advice that MS have published - specifically to not edit, save or transfer files on your homeserver within one on the listed applications (and, actually I’d say *any* application to be safe). Any files that have been corrupted will not be readable in any application.
Once again, from everything we’ve been told as MVPs (and the last update I had on this personally was the week before last directly from the Home Server team), very few people have hit this bug. We are all at risk, especially if we don’t act on the advice given (which I agree, completely sucks) and yep, we shouldn’t have to act on this kind of advice, but whilst we’re all free to have a pop at Microsoft about this (and please do let your feelings be known, that’s what this community is here for), let’s make sure we do it knowing our data is as safe as poss.
Cheers
Terry
March 10th, 2008 at 9:24 pm
Perhaps they should remove the product from store shelves until it’s fixed. I’m mad enough at having wasted my church’s money setting this thing up. I’d hate to have the list of aggravated people grow beyond it’s current point.
March 10th, 2008 at 9:46 pm
This is astonishing: Microsoft finds a bug that undermines the core functionality of their product - and then goes on to tell the users to (1) stop using that core functionality, and (2) sit quietly and behave for another few months. The “it only happens rarely/few users are affected”-argument is not much of a comfort and it seems to me that the only reason why Microsoft might get away with this that people have simply accepted lower standards for software products than for anything else they buy. How about buying a safe that will occasionally (but rarely) incinerate everything within it? Or a car whose breaks will occasionally (but rarely) cease to function? In either of those cases, the response would be an immediate product withdrawal and a compensation/exchange for whoever bought it.
But there you go,
Morten
March 10th, 2008 at 9:47 pm
Whoops, make that “whose BRAKES will occasionally…”
March 10th, 2008 at 9:48 pm
The bit I like best from the WHS Server blog is this “or people incorrectly testing the limits of their home servers.” What do they mean by that? If I thump the server with a whole lot of data I expect it to work. The other bit from the article that I think MS should clarify is whether using File Explorer they do suggest using it to do file copies or the command line but don’t list it above in the list of applications that are not affected. Though assuming that they are asking people to use one would assume that it’s safe.
Mind you, I have not experienced any data loss due to this issue even though I have also been using the server (multi drive system) in a careful manner ever since I first got it. The only thing I might have got tripped up with was iTunes. I run a range of stuff on the server including SageTV and have not had issues.
The one issue I have had is a lot of problems with copying some larger files to the server (e.g.: in the 4-6 GB range) from my home PC. This seems to work some times and other times fails (LAN net copy not wireless). The symptoms are the same with the file starting to copy and then slowing down and finally failing with the network share breaking. However the same copy from my laptop to/from the Home PC does not experience this issue across the WiFi network. This issue I can reproduce reliably.
Still file shares are now R/O and will only be turned on when I need to put data there (using File Explorer).
Overall very happy with the system but very disappointed about the possibility of corruption and sick of my friends who have other operating systems hanging shit on me all the time over it…
Cheers, Joffa.
March 11th, 2008 at 3:18 am
Well, too bad, but I’ve decided to be pragmatic about this. My personal data-security-measures involve synchronizing all the shared folders with an external drive every night. This enables me to keep off-site backups whenever I leave home. Since I was made aware of this data-corruption issue I have edited the files on the external drive only. It’s connected through an eSata-interface so I have no performance issues related with this practise.
I’ve got one question which is crucial to me. If the copies on WHS are corrupted then the synchronized copies on the external drive will be garbage too. So, does the corruption-issue depend on whether or not the shared folders are duplicated or not?
I
March 11th, 2008 at 8:18 am
Hi Terry,
Thnx for the answer….but - correct me if I’m wrong- you didn’t answer my WMP 11 question : can I safely PLAY MP3’s with it (I’ve told WMP 11 to use the \\WHS\MUSIC-share) without corrupting the files.
Secondly : I’ve always adviced people never to edit files on a share (even at work, I always copy my work-files local before editing them, and we have several “full” Win 2003 (file)servers there), because the longer the bread-trail (pc, network, server, …) the more change there is for a corruption in any file….and not only on WHS.
But in general, I think there are some places left in Siberia for the devs/testers/architects/… at MS who managed to screw up this wonderfull product. I’m a developper too, but what I do not understand is that, with all the resources at MS (Windows team, Server team, …), It will take them until june to fix this. They probably think : “what the heck, it are only a few early-adopters who are affected”, but what if - in the future- they are bringing this to “pro”-level ?? Givin the current situation, I don’t think that any CIO, server admin or who ever will use this in their server park…and than MS is probably going to say “we don’t understand why nobody is using this”…In my opinion, this is a potential nightmare for MS, with for more repercusions then they probably think off.
Greetz
Geert
March 11th, 2008 at 8:20 am
correction : “..the more change there is for a corruption in any file….and not only on WHS….” should be “.. the more chance there is…”
March 11th, 2008 at 4:31 pm
I still believe in WHS, it’s a wonderfull idea. Having said that, I believe MS should stop selling WHS until this is fixed. Potential buyers should be protected from this bug because I’m sure that those buyers are not aware of this bug or how this bug works. I was one of those who was affected by this bug, and I know how to reproduce it (It’s not that hard to do).
However, at the moment I have my drives out of the pool and added then manually just to be safe. Until the fix is released I’m not using the pool and the default shares. My data is a littlebit scattered, but it’s workable for now (and more inportantly: more safe!).
March 11th, 2008 at 5:01 pm
I fail to understand why they didn’t just automate the built-in Windows Server LDM and Dynamic disks. BAM–problem solved. Instead they write some file system filter driver and treat the drives as individual disks? Why?
Look at Windows Storage Server 2003… MS already had a product that provided 98% of the functionality of WHS and did it reliably. All they needed to do (IMO) is polish up the Web UI and they would’ve had a product ready to ship.
March 12th, 2008 at 1:53 am
I am incline to hold until late June if after that if they fail to fix it at that date then they should and MUST offer to upgrade to their robust and proven server but with keeping the same functionality as WHS with the Add-On side as I am like GaMeR I love WHS what the difference it made for our Home Network and a more enjoyable experience, I would compare WHS to what the Router did for me, opening the sharing in my house like it did open the sharing of high speed internet connection.
If they put the effort and men hours and make it a priority in the corporation it is possible.
I mean this is brand new market here and we are the one that will celebrate or quit that MS Home Server.
I was looking at being a beta user but for now it is not even a question because I can’t afford to tell my wife that our non beta server had just lost our family memory and all our encoding video file, this will be a real blow to me as it was my point to make the switch to a server instead of buying HD over HD for each of our PC and beginning to play safe with our PC knowingly that if we “screw “around” in our own “PC” that the data would be safe on the server side with the benefit of fast reinstallation from a network.
I won’t tell you how I fell right now because life is too short but I don’t need that at all trust me I really don’t need that period, meaning I thought with Vista and Server 2003 build I was safe but now I fell I am back in the game of 98 OS reliability.
My question is that for now do my “backup” are safe in a multi-hd drive or they are at risk as my share folder?
March 12th, 2008 at 9:10 am
Maybe we should launch a sort of petition or a blog to centralize our complaints. With a lot of referencing and search engines crawlers pinging. Maybe this way, we’d hope some action (i mean a real commercial gesture) from Microsoft. Because, correct if i’m wrong but, except in enthusiasts’ wesites like this one, nobody seems to be concerned or shocked by a “supposed-stable-server-os” release with a HUGE bug in the very central functionality it was sold for.
March 12th, 2008 at 10:22 am
Sooo…. can anyone confirm my data is safe if I just stick with an HP MediaSmart with a single 500GB disk? That’s my plan until the fix is released. It will be a bit of a squeeze, but 1TB drives should be much cheaper by then
March 12th, 2008 at 4:11 pm
I’m afraid I’m not clear on one point. This bug obviously impacts apps running on the client trying to save data to the server. Does this bug affect apps running on the server, saving to the server? Are add-ins and other server-based software at risk?
March 12th, 2008 at 4:45 pm
Geert,
I’d set the WMP client computer to connect to the shares with an account that has read only access, then use a full rights account to do file management tasks.
Mike
March 12th, 2008 at 8:24 pm
Zilch - single drive systems are completely safe from this bug.
AnotherTerry - the bug hits when you save/edit files stored on your home server on a home computer, rather than directly on the server itself.
Cheers
Other Terry
March 12th, 2008 at 9:15 pm
Hi Mike,
Thnx…I’ll do that
greetz
Geert
March 13th, 2008 at 12:11 am
Thanks Terry. Getting a single drive system and backing up everything to an external harddrive is my current plan (I wanted to have an offsite backup anyway). Can you tell me if there is a procedure replace the 500GB system drive with a 1TB one in the meantime? (ie Clone the system partion)
March 14th, 2008 at 2:12 am
I’ve been patiently waiting for this bug to get fixed before I make a commitment to WHS. So I’m using the “120″ day trial… Terry, has MS indicated whether they’ll “extend” the trial versions? (Mine indicates an expiration in July… )
And no, I’m not condoning MS for not fixing the problem, but hey, at least they admit it’s there an are working on it. Some other vendors refuse to acknowledge a software bug such as this and will only fix it in the next “upgrade” version (which you must pay for).
May 9th, 2008 at 11:33 pm
There seem to be a number of different opinions on this website as to the seriousness of the bug, and whether or not us early adopters should put up with it. Frankly I think its appalling, as I have a product that I purchased to make my files more easily accessible and secure, and in fact i have the reverse. However its not just the bug thats the problem, other core functionality, like being able to back up to an external device which will be added in Powerpack 1 that is also a concern. Finally the time take to solve the problem suggests to me that its a bigger issue than Microsoft are letting on. As I have a HP MediaSmart with 2×500Gb I don’t know why they don’t just issue a patch that lets you use it with Raid instead of this Drive Extender that is causing the problem.
May 11th, 2008 at 11:11 pm
WHS,
What a waste of time. Leave it to Microsoft to screw up another potentilly good idea. It is expensive, works when it wants to, and corrupts your data. Sounds like Microsft to me.
When will everyone figure out, the easist set-up out there is Windows Meda Center 2005? It works great as a server. It is a fully functional copy of windows, that you can do anyting you want with. Add Hamachi to the mix, and you’re set. As for remote access, just use TightVNC. You will need XP Pro, or Vista Ultimate to access your home systems anyways. As for sharing, Network Magic works like a charm. The only thing you don’t have is automated back-ups. I’m sure if one looks hard enough you can find a FREEWARE program that will take care of that for you.
My set-up:
Asus 775 Mobo, Core 2 duo (2.4ghz), 3 gb ram, 7500gt, 2 Happagure HDTV Tuner cards , 4 500gb Seagate HD’s in JBOD, Beyond TV.
works as my HTPC, and My media server all in one, plus I CAN stream to my Linksys Media Center Extender, without having to suck up resource on my normal pc.
May 11th, 2008 at 11:17 pm
“Michel Says:
March 12th, 2008 at 1:53 am
I am incline to hold until late June if after that if they fail to fix it at that date then they should and MUST offer to upgrade to their robust and proven server but with keeping the same functionality as WHS with the Add-On side as I am like GaMeR I love WHS what the difference it made for our Home Network and a more enjoyable experience, I would compare WHS to what the Router did for me, opening the sharing in my house like it did open the sharing of high speed internet connection.
If they put the effort and men hours and make it a priority in the corporation it is possible.
I mean this is brand new market here and we are the one that will celebrate or quit that MS Home Server.
I was looking at being a beta user but for now it is not even a question because I can’t afford to tell my wife that our non beta server had just lost our family memory and all our encoding video file, this will be a real blow to me as it was my point to make the switch to a server instead of buying HD over HD for each of our PC and beginning to play safe with our PC knowingly that if we “screw “around” in our own “PC” that the data would be safe on the server side with the benefit of fast reinstallation from a network.
I won’t tell you how I fell right now because life is too short but I don’t need that at all trust me I really don’t need that period, meaning I thought with Vista and Server 2003 build I was safe but now I fell I am back in the game of 98 OS reliability.
My question is that for now do my “backup” are safe in a multi-hd drive or they are at risk as my share folder?”
I would compare WHS, to Windows ME in terms of OS reliablity.