How To Read A Difficult Book

Have you ever had trouble reading a book about testing, although you consider yourself a big reader, a passionate tester and a curious person? I have.

I actually failed reading this particular book twice in the past two years. Well, maybe I didn’t fail reading it, but I definitely did fail understanding it and to learn the lessons that it offers.

I don’t even remember when I first heard about Jerry Weinberg’s An Introduction to General Systems Thinking (ITGST). It was a long time ago. I am always on the hunt for new books to learn testing and people I respected frequently recommended this book to me. They said they have learned something new from it each time they read it.

I was so excited to get started, the first time, that I got the book in pdf and began reading it immediately after the purchase, on the bus, when travelling to work. But after completing the first few pages of chapter one and flipping through the rest of the book, it was clear to me that this was going to be a challenge.

Why was it hard to read? It’s not just that there are a few English words (my native language is Czech) on each page that I don’t understand. It’s the way that the book is written. The sentences are long, and Jerry uses an academic tone. The subject matter is indeed systems thinking– applicable to general situations– but the examples that Jerry uses to illustrate it (at least in the first chapter) are mostly related to physics and computation, which means that in order to have a good understanding of it, I also have to be familiar with those topics.

I ignored those obstacles to start with and pushed through the first two chapters. But then I felt like I learned nothing from the book! Worse, it was like my brain turned off each time after reading a few sentences.

Since this was the first time that I encountered such a book and given that at that time I was 23 and just celebrated my second year of being a professional software tester after leaving the university, I came to the conclusion that maybe I’m not experienced and educated enough to read it just yet. I haven’t given up on learning about systems thinking, but instead, I switched my focus to other books such as Thinking in Systems by D. Meadows and Systems and Models by H. Bossel.

I put Jerry’s book aside and decided to come back to it later.

After a year, I gave the book another try. This time, I went through the first chapter and underlined each word that I did not understand; then I translated those words, typing it in my native language right into the book. Instead of reading a whole chapter at a time, I only read about ten pages. However, it did not help very much – in retrospect, I think this was partly due to my ignorance of learning more about the physics topics, before trying to make sense of what the book was trying to showcase with those examples. And ten pages were still too many at one sitting.

So the book with the treasures ended up being neglected in my library, again.

This year, I got interested in the applied epistemology topic from ‘A Testers Syllabus’ by James Bach. Since I did not know which resources (books in particular) to use to learn more about it, I contacted James personally. We had a chat about what epistemology is and how it relates to testing. He recommended three books to me – The Philosophers’ Toolkit by Julian Baggini; Gödel, Escher, Bach by D. Hofstadter and the most important book to him, he said, was – surprise, surprise! – An Introduction to General Systems Thinking. But, in my mind, the chances of learning more about epistemology just dropped to the minimum. How was I going to learn anything about this topic when I already failed reading that book two times?

I shared my story about the problems I had encountered when reading the book with James, and he said that he himself had read the book in increments of a few pages at a time, followed by thinking through its implications. He said he got so many ideas when he read it that his mind would overheat. Maybe the problem I was having with it wasn’t about me, but rather an effect of how much Jerry had tried to pack into the text. So, James suggested that I should also take it in small pieces and then discuss the content with him. I was hesitant for a second because I was afraid of failing again. But this was my best chance to finally succeed.

My Process for Reading ITGST (Third Attempt)

1. 1. I read one to three pages in a sitting.
2.  I underlined each word or phrase that I did not understand and translated those words or parts.
3. I highlighted phrases and sentences that I considered crucial.
4. I read each sentence multiple times, making explanatory notes in the margins about it (also in my native language, when needed).
5. I made an attempt to understand the topics that were used to illustrate the systems thinking problems such as the square law of computation, mechanics and Newtons’ laws of nature.
6. I wrote a summary of each subchapter after I read it. When having trouble with understanding the bigger picture, I waited a day or two after reading this subchapter and then came back to it and read it again, and only after that wrote down the summary. I also thought about how insights can be applied to testing.
7. After having written my summary, I sent it over to James. He read the same part that I had read and he commented on the inconsistencies between our understandings. This usually turned out into a discussion which follows.

Discussion Examples

Example 1 – ‘Complexity of the World’(pages 1-2)

Klára’s summary:

Precise mathematical results don’t necessarily have to matter unless it ties to some proof. These measurements are more often used to manipulate than to use them to prove something.

Mathematics can be used to describe a phenomenon in a way that might seem spectacular, but unless there’s the logical proof behind it, it doesn’t have to be very helpful.

The problems that science is able to solve have consequences that humans are not able to predict. That relates to the progress made in the sciences, where the knowledge came in very slowly during the ancient times (and before ancient times), in the range of thousands of years, but has sped up from that point on. We know that by the human actions, there will be some second-order effects (consequences) to the existing state, but we’re not able to predict if it’s going to be a change for the better or for worse and the precise magnitude of the change.

The purpose of general systems thinking is to help humans reveal the complexity of the world.

James: What do you mean about proof? In other words, what do you think that is about?

Klára: I thought they meant formal proof. Mathematical. But I think it could also be a different kind of proof than just formal proof.

James: Let’s say you have two apples and you add two more apples, how many apples do you now have?

Klára: Four.

James: How did you figure that out?

Klára: By applying the sum operation. 2x+2x=4x where x is an apple.

James: Is that a precise mathematical operation? It looks sort of like one, to me. Is it exactly 4? is it 4.000000000? Seems like that is what your operation gives.

Klára: You did not mention floating or double-precision so I went with integers. The answer is exactly four.

James: Okay, then. So…. you said that doesn’t mean anything without a proof. What’s the proof? Did you think about proof? Did you start with proof?

Klára: No

James: Are you saying you don’t agree with the idea that proof comes first?

Klára: No, it doesn’t come first.

James: Do you think you understand what the writer meant when he wrote that?

Klára: No (smiles)

James: I suspected that you didn’t. Because you copied his word “proof” instead of putting that into more useful terms. I will rephrase what he’s talking about. “It is pointless to calculate things when you don’t even know what you are talking about.”

In terms of the apple example, I asked you to add two apples to two apples, but you made a bunch of assumptions about what I was talking about. You didn’t check these assumptions you are apparently not worried about misunderstanding my question. (Which is okay, of course, unless it’s the sort of question that really matters.)

But the assumptions might matter. For instance, if you were to add two anti-matter apples to two ordinary apples, the result would be zero apples (because they annihilate each other). Or if you add two apples to two other apples that had been sitting there for 3 million years, you would have only two apples (because the old apples would have disappeared).

Klára: Yes, I was actually thinking about what kind of apple that could be (the fruit, or some reference to Apple’s devices), but I didn’t ask because the result I was thinking about would not change anyway. I didn’t think about the rest of it, unfortunately…

James: Whether you thought about all the different possibilities doesn’t really matter, in this case. What matters is what you actually were thinking. All I’m saying is that your equation is predicated on certain premises that were in your mind. Your answer is true within those limits. Thus, in order to communicate usefully and think effectively, you need to know the context of the numbers and equations that you use.

In summary, this is how Jerry is starting out his book: we need to understand the structure of a system in order to reason effectively about that system. The structure dominates all other things. A small change in structure may create a huge difference in behaviour or nature or appearance of evolution.

Klára: So what is the structure?

James: My most general answer (based on reading ITGST) is that structure is any pattern that persists in time and space.

Klára: Is it just one pattern, or does it include different ‘sub-patterns’ too?

James: Any pattern. A set of patterns is a pattern. A pattern may consist of a set of patterns.

But there is a more specific meaning in this case. A system consists of a set of things in a meaningful relationship with each other. (I’m using my personal definitions but I believe they are consistent with what Jerry wrote.)

So the structure of a system would mean persistent patterns in the meaningful relationship of elements of that system to other elements. That may include relationships of a multiplicative or exponential nature. Thus, a small change in one part of it could disable or magnify any given behaviour. This is just like software! You are used to this.

You could read the first chapter as implying that you can’t assess the quality of a software product just by carefully counting the pixels on a window. You have to understand the meaning and the relationship amongst the pixels, what drives the pixels, you need to know the functionality, perhaps the code itself.

Klára: That did not occur to me at all, but it makes sense now.

James: He explains in those pages that knowing how to identify and navigate patterns and relationships in and among systems allows you to learn more quickly. Try re-reading the first pages with that thought in mind, I bet it will make more sense to you.

Klára: “Measurements and equations are supposed to sharpen thinking, but, in my observation, they more often tend to make the thinking non-causal and fuzzy.” Could “measurements and equations” be changed to “automation tools” in the context of software testing?

James: Yes. or even more broadly, “best practices.” Any adherence to practice that becomes separated from reason and instead driven by habit.

Example 2 – ‘Mechanism and mechanics

Klára’s summary: The success of physics as a science is so great because it reduces the set of all the possible conditions that could be applicable to some object to only some conditions, which are sufficiently defined and under which the behaviour of the object evinces regularities.

It does not make sense to try to compute all the equations (even by approximate methods) of mechanical systems that have too many parts, because we don’t have enough time to solve it.

For that reason, we have to reduce the big mechanical system to a more simple one.

The importance of the parts of a system depends on the context of the question that the scientist wants to answer. Even if a certain part of a system is very small compared to the others, and it might seem insignificant to some, it might actually be the most important one for a certain scientist that deals with a context proper for researching this smallest part.

How this relates to software testing:

1. Observing the regularities in the system at first might help to learn about the system; what it does, why and as a starting oracle for future tests.
2. When engaging with a new product, it might be helpful to map out the features and behaviour, but not a lot of time is needed to spend on the subtle details that relate to an area that is probably not that important.
3. Start informally, formalize the testing later (if needed).
4. The importance of various areas of the system is dependent on the context – to what questions are we seeking answers to?

James: Just as a tree has a trunk, which separates into limbs and ever-smaller branches and twigs, we can look for the “trunk” of a system we seek to test— the core part. So we need to be able to identify the stuff that matters more and the stuff that matters less. Identify the trunk first, then you can deal with the lesser details.

Klára: Exactly, but apart from that, the importance is also affected by information that we gather from the stakeholders or developers, right?

James: Sure. There are different factors that go into what matters most.

Klára: I was wondering if this part:

“The success of physics as a science is so great because it reduces the set of all the possible conditions that could be applicable to some object to only some conditions, which are sufficiently defined and under which the behaviour of the object evinces regularities.”

Is also applicable to test techniques. I think it is. I mean, that various test techniques are useful only under some specified conditions.

James: Definitely! That’s the same concept I was just talking about. Equivalence classes are all about that.

Klára: “Many philosophers thought, with Laplace, that given precise observations on the position and velocity of every particle of matter, one could calculate the entire future of the universe. Although they realized that they would need a large computing machine, they lacked even the smallest computers”

What do you think is meant by ‘calculating the entire future of the universe’? The only thing that I think it could mean, is that the scientists could calculate the bodies exact locations at a particular time?

James: Not only exact locations but the result of every chemical reaction. In other words, the exact state of the future universe at any moment in time, as if running a video in fast forward.

This is a thought experiment, of course… it’s not physically possible, but depending on one’s model of the universe, it is or is not possible in principle.

Klára: “Newton needed all the simplifying assumptions, explicit or implicit, he could get away with, just as physiologists and psychologists do today.”

How do we draw a line to what’s an important thing to consider and what not?

James: The things that are important to consider are the ones that make a difference.

Klára: But we don’t know about all the things that could make a difference upfront right?

James: Okay, so what is your real question? Are you asking how to know about all the things that could make a difference?

Klára: No, I don’t think that’s possible.

James: Then what is your question? I mean, are you just trying to confirm that we can’t know?

Klára: Yes.

James: Technically, we can’t know for sure, but usually that doesn’t matter very much. You make calculations all the time without knowing things for sure. For instance, you calculate how long it will take you to get to work. When you do that, there are millions of tiny tiny tiny unknowns but you don’t worry about them.

A better question is not “how do you know,” but rather “how do you decide” what matters?

And the answer to that is theory. You have a theory about the world and you modify that over time. This theory tells you what variables matter. Theorists are people who build concepts that help practical people get work done, but we are all theorists to some degree.

Klára: So in testing, what is this theory? Is it the mental model?

James: Well tell me, if you test emoticons on Skype, do you think it’s important to vary the kind of tea you are drinking at the time? I bet you don’t. But why don’t you? Your theory of the world says that what you happen to be drinking cannot affect the behaviour of emoticons in Skype. This is not simply logic, it’s using logic in conjunction with theory.

Klára: That sounds more like an experience than theory.

James: Experience is not theory, at all. Experience is what happens to you. People often use the word experience to speak of the learning you have gained from what happens but that learning does not experience itself.

Klára: Exactly, the experience is different than theory. what I’m saying is that the situation described with the tea seems more like based on experience than on theory – I observed that the variation in nonalcoholic drinks does not affect my testing.

James: That’s theory! When you say experience you are just telling me where your data came from. That data has no theory “in it.” Nothing about drinking tea tells you directly that it doesn’t affect emoticons.

Klára: So what is ‘theory’ then?

James: Theory is a set of concepts that tells you what matters and how it matters.

Klára: Oook.. That makes sense now.

James: You have a theory that tea doesn’t matter. And that, in turn, is based on your theory of matter, energy, computers, etc.

Klára: Yes, I agree now.

James: Does the exact position of the sun affect your testing? I once experienced that sunlight affected a test I was running. When I realized it was happening, that wasn’t just an experience that showed me it was possible. It was an experience that made sense in theory, because the mouse had an optical sensor in it and there was a crack where sunlight could get through and the sun was shining through my office window. Light shining on a sensor designed to detect light is bound to affect that sensor, which is bound to affect the computer connected to that sensor.

Klára: That’s a good example.

James: Here is the big difference between experience and theory; you don’t have to experience the thing I just spoke of in order to accept that it is plausible. Because you already know about sunlight and sensors and computers that read sensors. I told you the bare details of it— the details that mattered. I didn’t tell you about the kind of grass outside my office window and you don’t feel the need to ask about that.

And now you know why propaganda and “fake news” and conspiracy theories can be so powerful, because when a story sounds plausible (i.e. when it fits your theory) and when a person wants that story to be true or is fearing that it might be true, then that person may be eager to believe it regardless of their experience.

Conclusion

As you can see, the discussion is an integral part of my current process of reading the book. That’s the point where I learn the most. Even if I do a really good job figuring out what Jerry meant with his words, I gain much more insight when discussing the content with another person – especially when the person is James, who had spent time with Jerry and is able to discuss on my follow up questions on the systems thinking topic, the answers to which Jerry did not explicitly state in the book (such as his definition of ‘structure’).

You will find that the benefits of this social method of reading will be different, based on who you read the book with;

• When reading the book with an expert, he can guide you through the material and provide valuable insights that will help your understanding of it.
• When reading the book with a fellow colleague or a friend who is new to the topic of the book just like you, your discussions might not be as deep as when you’re discussing it with an expert who’s able to switch to lecture mode, but you will still get some mental exercise by putting your understanding of the material into your own words and it can help each other’s energy.

Additionally, the novice readers might find it hard to discuss the books content on the spot without having written down a summary of at least some notes, that they can refer to during the discussion. If the language of the book is not their native language, this can cause problems as well.

Going through this process of reading the book with James, I found out that:

• There are different factors that influence the level of knowledge and the benefits that one gets from reading a book and speed is just one of them – there actually needs to be a balance between the speed of reading and the depth of understanding and knowledge; (in other words, there are books that take time to read and the fast speed of reading is not always the most important factor)
• While it’s almost always the case that discussing the content of a book with other people helps, for harder to read books like ITGST it is even much more important to do that