Updated bug list:
No book on testing would be
complete without a bug list. HWTSAM is no
exception! Some "book people" call this stuff errata, but let's just
call a bug a bug.
Here's what we know about so
far:
1)
In the intro, I state that Ken approached me in the fall of
2007. On reflection, it was actually late fall 2006. I began writing the first
chapter in January 2007 (chapter 4 to be exact). Minor issue, but an Author Bug
nonetheless.
2)
The table on p. 19 is missing a line between USA (California)
and Hyderabad (Production Bug)
3)
On page 95, where the header says. “3BV”, it should say “3BC”
4)
There's a bit of weirdness with the code coverage data on page
119. Bj explains it in detail on
his blog here. (Author bug)
5)
On page 120, the explanation refers to variables names myString
and myCharacter – but we use different variable names in the code (s and c).
(Author Bug).
6)
On page 123, the title of figure 5-6 is incorrect (the function
is SimpleGetNT5ClientVersion).
7)
On page 125, the text refers to the Int.Parse() function, but it
doesn’t exist in the code fragment (oops - Author Bug)
8)
On page 202, the text
reads “Total fixed found / total bugs fixed”, but it should read “Total
bugs fixed / total bugs found”
9)
On page 221, the “Positively False” sidebar is not a sidebar, and in fact the box around it
encompasses the additional points (Supported platforms, Complexity, and Other
factors) and section (We don’t have time to automate this) that aren’t part of
that sidebar (Author Bug)
10)
On page 226, the sample using InvokePattern could use some more
documentation (Author Oversight)
11)
In the code sample on the bottom of page 235, I have mismatched
quotes in two of the lines. (definite Author Bug).
12)
The story about testing on page 227 has the phrase "as
shown in the following graphic" For technical reasons, we pulled that
graphic, but missed that bit of referring text (Author Bug)
13)
On page 236, the method in the code snippet returns the value of
the testPass variable, which is not declared/initialized in the method itself. The
variable should be testResult instead of the testPass.
14)
On page 270, the seconde bullet at the top of the page should
say “…will solve”, instead of “…with solve”
15)
On page 292, we frefer to a “subtle” issue on line 19. That
issue is actually on line 20.
16)
Figure 14-2 on page 327 reads "Browsers (IF, Firefox,
Mozilla)". The IF should be IE or Internet Explorer but not IF (Author Bug)
17)
On page 331, the sentence “They key to success in testing…”
should be, “The key to success…”
18)
On page 346, the diagram uses the word “develoment” – we meant to use the word development.
19)
Part IV starts on p. 363, but the section text at the top of the
pages changes to part IV at the beginning of chapter 14. (Production Bug)
20)
In the index (p. 413), Königsberg has garbage characters in
place of the ö (Production Bug)
21)
We forgot to include a bibliography (or forgot to think
thoroughly about including one). Although we reference a lot of books via
footnotes in the text, one tester in particular is quite annoyed that we didn't
reference the writing that inspired us in the first place. I'll post a
bibliography sometime and include one in the 2nd edition.
Be the first to rate this post
- Currently 0/5 Stars.
- 1
- 2
- 3
- 4
- 5
A month or two back, Amazon began sharing BookScan information with authors. I can now see how many copies of How We Test Software at Microsoft are selling (US only, and Nielsen BookScan only tracks the large-ish book sellers) ? but it?s still sort of cool.
I can?t help but wonder who in Houston bought a copy, or why 4 copies sold in Washington D.C. last month.
I can also track ranking (only as far back as October 2009, I?m not as interested in sales ranking (the book seems to bounce quite a bit, but it?s still interesting to look at history. Sales rank from October 2009 to now looks like this:
Sales have been steadier than the sales ranks, and as such I expect that Amazon sales rankings are not based entirely on actual sales (links, page views, etc. all contribute (I think) to amazon rating).
Anyway ? not sure what to make of it, but I?m pleasantly surprised to see that the book is still selling steadily. I think if we sell about a bajillion more copies, I can justify a second edition.
Be the first to rate this post
- Currently 0/5 Stars.
- 1
- 2
- 3
- 4
- 5
Amazon just started sharing Nielsen BookScan reports to authors. Since I don't get commissions for How We Test Software at Microsoft (hwtsam), I don't generally see sales numbers, but I'm happy to see that people still buy the book (note for internal readers - orders through msmarket do not reflect in the BookScan ratings). Anyway - it looks like the book sold a whopping 32 copies last month. That's certainly not a big number, but it's sort of nice to know that people are still taking a chance buying a book about testing at Microsoft (also note that BookScan numbers are US only).
Today is the two year anniversary of the release of hwtsam. I wrote a post a year ago to commemorate the one-year anniversary, but I don't think I'll look it up to link to it because that would just make me too afraid to repeat myself.
I should take this moment to point out that Michael Larsen is in the middle of a chapter by chapter review of the book. He?s doing a fantastic job summarizing the book and sharing examples. Thanks Michael.
I remain proud of how the book ended up. Yes, it could have been better, and yes, I do find it slightly ironic that a book on testing was released with a small handful of typos, but I do believe the points still get across.
The book was a snapshot in time - ok, maybe not a snapshot - more of a window. If we wrote the book today, certainly, some of the subjects would change (and certainly my bio), we'd have more stories (or at least more current stories), but I'd guess that most of the subject matter would be the same - and I think that's a good reflection on our choices for which areas of Microsoft testing to highlight.
The stories were my favorite part of the book. I'm sure this is partially because I didn't have to write most of them, but I love books like "Writing Solid Code", and "The Pragmatic Programmer" that have stories and anecdotes buried within the prose and wanted to try something like that. I know that different people find value in different things, but I hope that some of the readers found value in the stories as well.
I recently put together a case study for an upcoming book containing a collection of test automation case studies (more details on this book when they're available). In my search for an "interesting" case study to document, I came across dozens of really cool things going on within the walls of Microsoft that people wanted to share. My first thought was "come on folks - don't you have a blog or something where you could share this stuff?" I'm not ready to shepherd another book yet, but someday when I find myself with a few spare months of time to blow, I may consider putting together a book of stories and anecdotes only about testing at Microsoft. We'll see.
I speak at our new employee orientation periodically. I'm supposed to give an inspirational talk about my product or something I'm passionate about. I always talk about software quality and why we need the new people to speak up and use their fresh eyes to improve our company - this holds true for the software folks as well as the sales, legal, marketing and HR folks in the room. After a talk last spring, a young woman ? a new college hire, tracked me down in the back of the room following my presentation. She asked me to sign her book and told me that she discovered the book in college and knew as soon as she read it that she wanted to be a tester at Microsoft. That situation has happened a few times since then, and the feeling of satisfaction on accomplishing one of my big goals in writing the book is huge.
In closing my annual Happy Birthday to hwtsam post, I want to say thank you again to those of you have read the book, supported the ideas, or even taken some of what we talked about and applied it to your own work or own organizations. You all made the effort that went into the book worthwhile.
Currently rated 4.0 by 1 people
- Currently 4/5 Stars.
- 1
- 2
- 3
- 4
- 5
In HWTSAM, we (Ken, actually) talked a bit about tester DNA ? that bit of mental goo that makes some people better (or at least more prone to being) testers. As I?ve been talking to (potential) testers lately, I?ve had a chance to dwell on this a bit more. What is it that makes a good tester? Given that the answer to that question requires context, I won?t answer that exactly ? instead, I thought I?d share some of my thoughts on what I look for in testers.
Testing is broad and evolving ? nobody knows everything about it, so the ability to learn quickly is critical (this aids in problem solving as well). If you?re the type that takes a long time to ramp up in new technologies or concepts, you may struggle as a tester. Probably more critical is a passion for learning. Good testers don?t wait for ideas to come to them, instead, they seek out knowledge - they not only learn what they know they need to learn, they find ways to learn what they don?t know (i.e. they strive to resolve second level ignorance). I believe that the big innovations in testing will come from applying knowledge from outside the field of software and software testing. In order to advance the state of the art in testing, we need testers who seek knowledge ? and who are able to apply those abstract concepts to solve some of our big problems in test.
But ? to take care of that last sentence, you?re going to need people who can see the big picture ? systems thinkers. There are numerous people who claim to be systems thinkers, but systems thinking takes practice as well as some innate ability (or DNA) to be beneficial ? and it?s much harder than many people think. Often when I interview testers, I ask a ?testing? question that has two parts to solve. The first (the question I actually ask them) is obvious and has a solution that is difficult enough that they solve it as they would any other question. However ? there?s a hidden problem in the question. The good testers quickly see the secondary problem as the far more difficult problem to solve and focus their answer on solving the underlying problem. These are the systems thinkers ? they know to look at the whole rather than the parts and know that understanding interactions and patterns are keys to good problem solving. The great testers ? and there are only a few of these ? can actually solve the problem reasonably well (frankly, I worry about testers recognizing the problem more than solving it, but I?m frequently impressed by testers who nail every aspect of this question).
nitpickers moment ? for those of you who will take this opportunity to gripe about SDETs, no part of solving this question relies on programming skills. It does require that you can think and see beyond the obvious. I?m not going to put the question on my blog, because I still want to use it. I would be happy to discuss it with you privately (or via an IM session) if you?re insanely curious.
There are numerous other skills I look for in testers, but I consider those to be supportive and of the ?more is better? category. For instance, if you are completely disorganized, you may not be successful, but you don?t have to be the most anal note taker either. You need some degree of organization, self motivation, confidence, and trust to be successful, but those really only show up on my radar if you truly suck at them.
As I re-read this post, I realize that the things I mentioned above are also the things that make testers successful in the long term ? so it makes sense that?s what I look for when hiring testers. And I think that?s good!
Currently rated 4.7 by 3 people
- Currently 4.666667/5 Stars.
- 1
- 2
- 3
- 4
- 5
I think it?s been a year since How We Test Software at Microsoft made its way to store shelves (and amazon). For the first few months, I watched the amazon sales ranking multiple times a day. I took a screenshot last December 18th that shows one of the few times we hit the #1 testing book. The book actually made up in the 7k range overall once, but apparently I didn?t take a snapshot.
Since then, the Chinese version was released, and the Korean version is imminent, and I?ve traded writing on weekends and evenings for more time with my family (and occasionally, more time for work). When I finished writing the book, writing another was the farthest thing from my mind, but since then, I wrote a chapter for Beautiful Testing, and have at least entertained the idea of writing something else?someday.
In hindsight, there are many things I?d like to redo with the book, but it is what it is, and I can live with that.It?s a book full of information and stories about how testers at Microsoft do their job. It?s a book about people, approaches, and some tools. It talks about when and why we automate tests, but covers a wide range of other topics as well, and I?m happy with the story it tells.
I think the book has sold somewhere around 5-6k copies (I haven?t looked at numbers in 6 months, but I?ll update this post if I do). That?s certainly not a huge number as far as books go, but it?s still amazing to me. My thanks go out to everyone who bought a copy (and more thanks to those who actually read it).
Currently rated 3.0 by 5 people
- Currently 3/5 Stars.
- 1
- 2
- 3
- 4
- 5
Thanks to everyone who attended my talk today. Feel free to fire more questions here if you have them.
Slides are available here.
Currently rated 3.2 by 6 people
- Currently 3.166667/5 Stars.
- 1
- 2
- 3
- 4
- 5
I am excited to see that the Chinese version of HWTSAM is out (and soon to be followed by a Korean translation).
You can find information here:
http://www.china-pub.com/196002
and on Amazon.
Currently rated 3.2 by 6 people
- Currently 3.166667/5 Stars.
- 1
- 2
- 3
- 4
- 5
My talk is over - I've received a lot of positive feedback (although the people who hate my talks never track me down).
I did have fun though, so at least one person in the room had a great time. My slides (which are nothing like the slides I submitted) are here if you'd like to take a look.
Currently rated 3.0 by 5 people
- Currently 3/5 Stars.
- 1
- 2
- 3
- 4
- 5
On request from Adam Goucher – another excerpt from How We Test Software at Microsoft. BTW – Adam wrote a review of HWTSAM here – although Linda Wilkinson beat him to the clever title.
This is from a section on quality in chapter 16. It’s something I believe strongly in and would love to hear your comments.
Many years ago when I would ask the question, “who owns quality,” the answer would nearly always be “The test team owns quality.” Today, when I ask this question, the answer is customarily “Everyone owns quality.” While this may be a better answer to some, W. Mark Manduke of SEI has written: “When quality is declared to be everyone’s responsibility, no one is truly designated to be responsible for it, and quality issues fade into the chaos of the crisis du jour.” He concluded that “…when management truly commits to a quality culture, everyone will, indeed, be responsible for quality.”[1] A system where everyone truly owns quality requires a culture of quality. Without such a culture, all teams will make sacrifices against quality. Development teams may skip code reviews to save time, program management may cut corners on a specification, or fudge a definition of “done”, and test teams may change their goals on test pass or coverage rates deep in the product cycle. Despite many efforts to put quality assurance processes into place, it is a common practice among engineering teams to make exceptions in quality practices to meet deadlines or other goals. While it’s certainly important to be flexible in order to meet ship dates or other deadlines, quality often suffers because of a lack of a true quality owner.
Entire test teams may own facets of quality assurance, but they are rarely in the best position to champion or influence the adoption of a quality culture. Senior managers could be the quality champion, but their focus is justly on the business of managing the team, shipping the product, and running a successful business. While they may have quality goals in mind, they are rarely the champion for a culture of quality. Management leadership teams (typically the organization leaders of Development, Test, and Program Management) bear the weight of quality ownership for most teams. These leaders own and drive the engineering processes for the team, and are in the prime organizational position for evaluating, assessing, and implementing quality based engineering practices. Unfortunately, it seems that quality software and quality software engineering practices are rarely their chief concerns throughout any product engineering cycle.
Senior management support for a quality culture isn’t entirely enough. In a quality culture, every employee can have an impact on quality. Many of the most important quality improvements in manufacturing have come from suggestions by the workers. In the auto industry, for example, the average Japanese autoworker provides 28 suggestions per year, and 80% of those suggestions are implemented[2].
Ideally within Microsoft engineers from all disciplines are making suggestions to improve quality. Where a team does not have a culture of quality, the suggestions are few and precious few of those suggestions are implemented. Cultural apathy for quality will then lead to other challenges with passion and commitment among team members.
[1] STQE Magazine. Nov/Dec 2003 (Vol. 5, Issue 6)
[2] The Visionary Leader, Wall, Solum, and Sobul
Currently rated 3.0 by 5 people
- Currently 3/5 Stars.
- 1
- 2
- 3
- 4
- 5