Why is RSpec So Hard? (Part II)

About this Episode

The panelists continue for part two of their discussion on testing code with RSpec!

✨ Check out Part I here! ✨

Mike kicks things off with a maze metaphor, illustrating the importance of navigating code branches. He emphasizes the branch-focused approach for comprehensive test coverage and cautions against excessive testing of external libraries while highlighting the significance of testing public interfaces.

The episode also addresses the hurdles newcomers encounter when venturing into the world of RSpec. Hailey and Eddy share their initial struggles, likening it to learning a new language. They agree that familiarity with RSpec's unique language and structure grows with time.

In the latter part of the discussion, Eddy and David share insights on RSpec's error messages. Eddy expresses frustration with cryptic messages, particularly involving hashes and scientific notation. David recommends using the --format documentation option in RSpec to enhance error message clarity and suggests writing descriptive context, and it blocks for improved test readability. Mike highlights the importance of enhancing error messages and proposes an unconventional approach to structuring tests while emphasizing the value of starting with the unhappy path.

All said and done, this episode takes you on a journey through RSpec, covering testing strategies, best practices, newcomer challenges, error message improvement, and intriguing insights into test structuring. It underscores the significance of clear error messages and offers valuable guidance for enhancing test readability and effective debugging.

Transcript:

DAVID: Hello and welcome to the Acima Development Podcast. I'm David Brady. I'm hosting today. We've got a really great panel. We've got Ramses, and Jonathan, Freedom, Mike, a couple of people snuck in. I think I see Eddy. Hailey just walked in. This is going to be a good episode. We're going to talk about how to organize RSpec to make it a little more understandable and how to deal with some of the difficulties in it. Mike has a thing to drop on us.

MIKE: And the way that I'd like to kick this off is to talk about a maze, either a maze or a cave. Visualize your pick. You're either going to walk through a labyrinth above ground, or you're going to walk through a labyrinth below ground. Either way, you're lost in endless passages that branch here and there. You know that you want to find your way out, right? There is a way out, but going into this maze or this cave, you don't know which way that is. And how do you solve that problem?

I'll make a proposal here of a solution, a straightforward solution. And here's my proposal: Every time you come to a fork in the path, you have the ability to mark it, right? And this is actually a very simple rule. One way you can do this is just to always go right [laughs] first, right? You always turn right when you get to a branch. You explore to the right. And then, once you get to another branch, you go right. You get to another branch; you go right. You get to another branch; you go right. There's even a name for this in the computer science literature of, you know, you're doing a depth-first search.

You're going to go down as deep as you can in the cave, right? You're going to go follow that all the way to the end. And when you reach the end, you backtrack to the previous branch. And then, you turn left, and you don't turn left down any branch until you've explored as far as you can to the right. And then, you can go and explore down that branch to the left. And then, again, you have your right rule where you're going on as deep as you can. You come back, and you backtrack. When you finally have to, follow the left. And you keep on doing that, making sure that both sides, both branches, at every point until you've explored the full maze.

I actually read about this once. I've thought about this since I was, like, a teenager because I read once that they design mazes for which this rule doesn't work. I'm like, wait, what? How? Well, the way that you design mazes doesn't work as you have concentric rings, which means the branches don't quite follow the right rule. Because if you have concentric rings and your solution is at the center, you can have a wall all the way around at the outermost layer. All of the turns are left. And if you turn left, then you're just going to be going around a ring again. So, at every point, the rule doesn't work. But you can make some minor adjustments to it to kind of make it work.

But in a traditional maze, like the kind that would be underground in a cave, they can't do that sort of thing generally. You have to follow one branch or the other. And why am I talking about this for RSpec? I say this because our goal when we are testing code is to get coverage. We want to make sure that everything works. And a lot of times, we think, well, my code is really sophisticated. It does really interesting things, which is probably true. When you break down those interesting things, in the end, they tend to be binary decisions. You go right, or you go left. It's a decision tree.

As you follow down on your code, at every point in your code, you're either executing something, in which case you're always going through it, or you have some sort of condition. And sometimes, I will say, sometimes those conditions are implicit. It doesn't always include an if clause, for example. A return sends something out or a break. Maybe it's not explicitly doing an if, but there's some implicit types of conditions. Either way, it's a branch.

At every point in your code, it's either going to go straightforward, right? It's going to keep on executing through instructions, or it's going to branch somehow. And if that's what your code always is, which it should be, it's either going to progress linearly, go through procedural step by step, or it's going to branch. And so, if you want to have complete coverage of your code, then what you need to do is simply cover every branch.

I'm going to stop on that for a second because I think it's a critical realization that's really helped me with RSpec. Because if instead of thinking about my code as a set of functions and maybe I've got some anonymous functions, and classes, and objects, which is all true, from a testing perspective, I think that your code is just a set of branches. I don't really care about your objects other than as entry points. What I care about from the testing perspective is coverage. So, I care about the branches.

And thinking about it from that branching perspective changes your mindset. You're exploring the cave, right? And you want to explore every branch. And once you've changed that from thinking about it as this sophisticated, beautiful labyrinth, which it may be, from a testing perspective, you should think about it differently. You should think about it as a bunch of branches. It's a tree with branches. And if you cover all the branches, your testing is complete.

I'm going to pause there for a second to think about the metaphor. Well, RSpec has tools to do that, but I'd like to talk a little about the metaphor first. That's how I kind of wanted to introduce our discussion. I had that idea in mind before we talked.

DAVID: Now I'm very excited about this because I'm either going right where you want me to go or I'm hopping into, like, a just a weird hair up my brain spark, which is that when I sit down with a great, big thing to work on, I get daunted. I get blank page syndrome where I'm like, ooh, how am I going to get all this tested?

In the maze world, I've heard this called the right-hand rule or the left-hand rule. And it's got an interesting property that if you put your right hand on the wall, then when you get to the branch, you'll go down, like, the right-hand side of the branch. You get to the room at the end of that thing. You walk around the outside of the room...or the inside of the room. You keep your right hand, and now you're leaving the room with your hand on the other wall. When you get back to the branch, you're now touching the wall between the thing you just went down and the next one over. So, you just keep your hand on it, right?

The thing that I love about breaking it down this way is that it's very, very clear where you start, which is, to quote Alice in Wonderland, "At the beginning," right? And then, what do I test next? You've always got a next. It's hard to know how am I going to test all of this? I don't know. I don't know. But I know what I need to test first. And what do I test next? Well, when I finish the thing I was on, I'm just going to test the first thing that's left. It's a lovely, little recursive thing. You always know what's next until you run out of next, and that's when you're done.

MIKE: So well said. I've worked with a lot of people over the years who've looked at their code, like, where do I start this test? Because it is daunting. It doesn't just apply to testing. It applies to a lot of things. But we're talking about the testing problem, in particular, and that's exactly it. Well, what's the entry point of your code? And let's start by creating a context for your first branch. And there is your first step.

And once you've explored that, you know, and maybe you have to do some nested contexts in there because that branch branches. And you keep on exploring those nested branches until you get to the bottom. And you come back out, and you're like, well, okay, what's the next branch? And I maybe need another context [inaudible 07:02] explores the next one. And that's it. And you can run through the test just like that. And there's always a next step. It's really easy to think about. And the only thing you have to think about at every step is, well, what's the other branch?

EDDY: You know, Mike, I have a question regarding coverage. When writing tests for that file, in particular, should we always be shooting for 100%? And if not, what are the exceptions?

MIKE: Well, that's a great question. Let me answer it this way. And we're talking about RSpec. RSpec is so named because there's, like, a specification there, right? It's a way of specifying what your code does. And if we have not fully specified what our code does, then it isn't very good documentation, and there's undocumented parts of our code.

DAVID: I'm going to come at this from a completely different angle. I'm going to say something...here's my hot take: You should never be striving for 100%. You should be striving for what's the most important thing to test. Or like Mike's saying, literally, if you're going through an order, you branch. I like the branch, in my mind, between the most important thing that's in front of me and everything else. So, like, when I'm testing an API call, my first branch is the happy path. I want when I do call the thing; I want it to work.

Then, what I will do is I will branch into the unhappy stuff. What's the biggest, most important unhappy path? That's my next branch, and the next one, and the next one, and the next one. At some point, I'm going to realize I've tested enough or I'm out of functionality, especially if you're doing RSpec and you go in and you literally spec the code. You've all heard of TDD, Test-Driven Development. You go in, and you say, I'm going to call this API. And when I pass it this user and this status, I'm going to get back a 200 OK, and that user will be updated.

And I love how expressive RSpec is for this because it makes it very, very easy to specify what the code will do, and you haven't even written the code. You reach 100% the same way, which is you're like, what's the next thing I need to work on? Oh, I'm out of functionality to specify.

MIKE: I was just saying if you have a cave system that has multiple entrances, those should be seen as separate systems. Your code has a public interface, public methods. And those are the entry points, right? That's the entrance to the cave. You should go into every entry point that is public, and that's it. If it's private, well, then it's not effectively part of the code that you're testing because it's unreachable. The goal here is to specify every branch that is reachable through the public interfaces.

I'll put one other caveat here, which is that we should explore the boundaries of our unit of code because we're talking about unit testing here, not integration testing. We should explore the boundaries of our unit of code, which means that you get something that leaves, say, the class that you're testing. That should be mocked or stubbed, so you don't actually go beyond that point. You don't ever leave your cave.

MIKE: Mike, I take back every bad thing I've ever said about you. That was beautifully put.

EDDY: [laughs] I was going to say because then you get in the realm of integration tests, and that's not what you want.

DAVID: Yeah.

EDDY: But specifically, I was going to say, what if you have methods that just do trivial logic, but it's still public? At that point, should you be testing something so trivial as to, like, display a text or, like, decorate a text [inaudible 10:08]?

MIKE: Well, if your code is so trivial that it doesn't need to be tested, then why did you write it?

DAVID: There's actually a good answer to both of these questions. The testing push came around, like Y2K, late '90s, like with extreme programming. And those folks...and they were coming from Java where you could specify accessors. So, you could very easily say, this is a read accessor on this. It's a public entry point. Therefore, it is behavior; therefore, it should be tested.

And everyone kind of stepped back and said, "Yeah, but this is a language feature. It's never going to have a bug. It's never going to be a problem." And they collectively decided this is the one case when we do not test public behavior because it can't fail. Or if it does, you've got bigger problems, or you've literally misspelled the method name.

MIKE: You said something that I think was critical, which is related to what I said about not leaving the cave. You don't test outside your cave. That is, if you didn't write the code, you don't test it. If it's the language or the framework that is providing the feature, it's not your code, and you shouldn't test it because then you're testing somebody else's code. You're not testing your unit of code anymore.

DAVID: Right. 100%, I'm embarrassed to say how late in my career I got before really having this hammered home because it was this morning. I'm writing a thing that talks to an API. And I have always considered the HTTP library, like Ruby's Net::HTTP library, that's framework code. That's core library code. We don't want to test into that. But my application is going to talk to Net::HTTP. I might want to specify that the conversation there is handled correctly.

I've been on project after project after project where we've said, let's not use Net::HTTP; let's use Faraday. Or let's not use Faraday; let's use HTTParty. Or let's not use HTTParty; let's use Typhoeus. And just on and on and on and on all the way down the road. And I ended up getting to this point where I would walk around the application out the backend, where the cables go off into the internet. I'd use, like, WebMock and intercept calls going out over the wire to HTTP over the wire. It's a little bit of an exaggeration, but you get the idea. I'm all the way at the other end of Ruby catching these things coming out.

And I was talking with my team lead. And he's like, "Why don't you just stub the call to Faraday?" And I'm like, "But...but...but," right? Because if we change that library, it's going to change the conversation on my specs. It's going to break. I actually had to step back and go; I didn't write Faraday. I didn't write HTTP. I am writing the code that has the conversation with whatever web library. And guess what? That is exactly the kind of thing I want to test. I want to assert that I'm having the correct conversation with whatever web library I'm using. And I want to specify that it's having the conversation with that web library. You don't want to send Faraday calls to HTTParty, right? It's not going to make sense.

So, yeah, like I said, a lot later in my career. It literally was this morning. I'm not exaggerating. I'm like, oh, oh yeah, I had a heuristic to make that call the other way around. And the trade-off there was that, yeah, sometimes I end up testing library code that I didn't write. And, [vocalization], boy, yeah, sometimes you run into trouble because the library maintainer can change their implementation. And yeah, you see the trade-off there, right? It just spooled out of control. You don't want to do that. Never test a Rails private method.

MIKE: And, again, let's make sure we're making the distinction between unit tests and integration tests.

DAVID: Yes.

MIKE: And I think that's where we get into a lot of trouble. If you want to make sure your system works with another system, you should have a small set, I would argue, because they're really expensive and slow, of tests to make sure that integration works, which is different from your standard test suite that you run over and over and over again because it's fast and effective, and it doesn't break all the time.

DAVID: Right. Right. I realized the other part of the trade-off is that coming from API land previously in my career; I'm used to being handed a JSON document of post this to our REST server. And we will give you—then they hand me another JSON document—we'll give you this document in return. And having that and being able to specify that exactly I want this many bytes to go out, and I want these exact bytes to come back, then it does make sense.

And, like you said, integration test. Yeah, you don't want to test 401, 403, 405, 422, 404, every single possible 400 error. You don't want to test every one in an integration test. These things take forever, relatively speaking. Maybe I fire you the happy path, and I stub out with a response. And then I fire out one or two unhappy paths, and then we're done. We walk away.

MIKE: So, I've talked a lot about my idea of how to test branches. For me, it's been kind of an epiphany [laughs] when I saw this, and it's relatively late in my career as well, not quite this morning but not five years in, to realize, oh, well, all I'm doing is testing branches. But, you know, having that different lens with which to see the world has dramatically affected testing.

And I've seen it really work well for other people as well. People who were just banging their heads, like, where do I start? Were able to very quickly progress by thinking about things through that lens of, well, what's just my next branch? And there may be others on this call who've approached things that way, and I'd be interested in hearing other people's experience with that approach.

EDDY: You know, I'm kind of curious in gauging people's perspectives here. Everyone on this call, I think, to some degree, has touched RSpec for the past couple of months. I'm curious to gauge what other people's perspective was the initial exposure that they had to RSpec. Did it give a facade of difficulty, and if it did, to what degree?

DAVID: I got into RSpec with version 1.0, which put the should method on every object in Ruby, like, throughout the entire space. Of course, in Ruby, even integers are objects that you can call methods on. And so, you could literally write three dot should equal three. That was the moment that test-driven development really lit up. And I'm like; I can write this before; I mean, there was nothing stopping me from saying assert equal three comma three prior to this. But, for some reason, in my brain, assert equal three; three is something you write after. You know, assert equal, A plus B equals five, right?

In my mind, that's something you do after you have written the code, and you're just trying to button things up and prove to the maintainer that it works. But, in my mind, A plus B should equal three. It pushes you to say, hey, write this before you write the code. We're not going to say it does equal three; we're going to say it should equal three.

And I really liked that RSpec, when they got rid of the should monkey patch—which makes me sad, but I understand why people get rid of it—I liked that they kept that future subjunctive tense by saying, we're not going to assert. We're going to expect. And so we say expect A plus B to equal three. It's a little easier for me to write that after the fact and see it kind of in both verb tenses. But it does let me hang on to that bright torch of, I'm going to write this, and then I'm going to go write the code, which I realize is tangent to the topic of today, which is making RSpec easier. But, you know, it's my baby, and I like to show it off.

HAILEY: I'll speak on my first experience with RSpec. I feel like it was probably just a few months ago. I think as interns joined in on The Skills Clinic, we were working on some advent of code. And we were, like, in the middle of the project. So, we didn't see, like, the spec being created. I just saw, like, a completed spec file, and I think that was really confusing for me. Because, like, I feel Ruby is similar enough to other languages that I was able to understand, okay, this is, like, a loop. This is an if statement. You know, I'm returning this section of code, things like that.

But words like describe, and let, and subject, context, before do blocks I hadn't seen those before. And so, that was all really new to me. And to just see, like, 50 lines of code of things that I hadn't seen before that was really confusing. And then, once we ended up breaking it down, and once I was able to write my own spec and see them being written, not just a completed spec, that really helped me understand how RSpec works.

DAVID: Nice.

EDDY: That was my same experience when I first picked up on RSpec to [inaudible 18:26] in QA. When I first started, I was like, wow, what is this whole other language, right? Like, how is the structure, and why is every one so opinionated? [laughs] And how this is written. So...

HAILEY: Right. It seemed like a different language.

EDDY: Absolutely. You got some overlap here and there, very subtle. But overall, yeah, it can be a bit overwhelming if it's the first experience with it. So, I think that's, like, sort of, like, makes it a little difficult for new developers who are testing in RSpec initially, right? There's, like, some sort of ramp-up to it.

DAVID: Yeah. And there are some surprising things that come from that, right? If you pick up, like Jasmine, which is a JavaScript testing framework inspired by RSpec, then it has context and describe, and it. It's a function that takes the name of the thing that you want to describe or specify. And then, it takes a Lambda of just the function you want to execute when that happens. And that's kind of what you see in RSpec, right? You see, describe, and then a thing, and then do/end. And the do/end is like that...it's a proc. It's literally a block of code that's going to get executed as, like, an anonymous function.

But it's really easy to just go, oh, it's magic, and I don't need to worry about it. And that's absolutely true 99% of the time. But eventually, you start wondering, how does this function ever get called? And that's when you start lifting up the tablecloth and looking underneath RSpec, and you start seeing that, oh, that it do it doesn't define a method. It defines a class. A what? Yeah, it defines an entire class, an anonymous class. Its execute method...I can't remember the name of it. Is it run? It might be a run () method, but it's got a method on it that will, you know, execute the test.

And when you run RSpec over, like, a directory, it picks up every one of these files. And that it do makes a class, it do makes a class; it do makes a class. And then those three classes are inside a context, which makes a class. And the definition of that class is define these three classes that we just talked about. Once they are all defined, and everything is completely loaded, that's when RSpec says, okay, you know, roll call, who all did we get defined? Okay, cool. Start at the beginning and start running your stuff. And it starts executing it through.

And that is why you can end up with, like, this really weird stuff, like, where you have a spec that fails, and instead of giving you, like, the name of the spec, you end up with example three, bracket five, bracket two, and you're like, what...what is this? Well, okay, it's actually you were doing it do context do without a name. You were just doing, like, it do. Like, you used subject instead of saying, you know, describe thing. You're just like, well, describe do. You shouldn't do that, but you can. And there are people that if you can do it, they will.

And [chuckles], Robert, if you're listening to me, I love you. We can still be friends. I had a co-worker named Robert. He could point anywhere in a file, and he could tell you what the bracket offsets, the indexing scheme that RSpec uses under the hood. The point that I'm trying to make...I'm actually not going to talk all the way through to [inaudible 21:14]. I'm just going to assume that we all got lost in that weird classes of definitions because that's the whole point. It's magic, and 99% of the time, you don't need to keep track of it.

But the entire point of this is there are people who will defend, like, Test::Unit, which is really, really old, Minitest, which is getting old. Minitest is very, very simple. The entire source code for Minitest used to fit on, like, two printed pages back when printing was a thing that people did in the world. So, it was very easy to understand everything Minitest does under the hood. And RSpec was just this document the size of, you know, the federal tax code. And so, trying to get your head around what it does was very, very hard.

So, I 100% agree. Most of the heavy lift that RSpec does is to create this domain-specific language, a DSL for testing. Well, not for testing, for specifying the way code should behave and then making it behave that way. As long as it works the way you want, you don't ever have to open it up. And you're like, oh okay, yeah, once I've learned this language, then I can use it. And it's very fluent, and it's very easy to understand.

But, Hailey, you are 100% right that I started that with once you learn that language, and it almost sounds like I said the word just right or easy. Or how hard would it be, right? No, no. I fully admit that that once implies not until, right? You have to learn the DSL before you can leverage it.

MIKE: But I will say something interesting about this. I'm going to kind of go back to the cave idea, this maze traversal, which is that RSpec is actually kind of opinionated. It lets you do a lot of things. But that syntax that it uses, those contexts, for example, encourage...just like Dave was saying about that should or that, you know, is expected or, you know, expect, there's some suggestion as to which way you should go.

RSpec doesn't have a lot of top-level methods, but it has describe. It has it, which is where your assertions go, and it has context. The workhorse of RSpec is the context. And a context represents a branch built into the framework. If you think, well, oh, I'm going to set up tests for a branch, and I'm going to give it an anonymous function, and conceptually and in practice, that's what's happening is you're saying this branch is going to get an anonymous function that will run the tests on it. And that's what RSpec is doing.

If you want to know what that domain-specific language is doing, it is giving you the ability to set up a test for a branch. And I think seeing that and understanding that's what it's nudging you to do can help overcome a lot of apprehension.

DAVID: Absolutely.

EDDY: I do want to say that ever since I started testing with RSpec, I've hated red dots forever.

DAVID: You absolutely should.

EDDY: Anytime I see a red dot, I just hate it, period. Number two, I think what makes RSpec really difficult, especially for someone coming in fresh, is the fact that the error messages could be a lot better, right?

DAVID: Yeah.

EDDY: Like, some of the messages that you get for errors, you're just like, what are you talking about? Like, what do you mean no_args expected, [laughs] right?

DAVID: I have a strong opinion about that. Finish your idea, and then ask me about the errors.

EDDY: I was just going to say, I think what really makes it really difficult to pick up initially is just that the error messages that are spitting out, like, unless you have context on that, like, it makes it extremely difficult to really debug, like, what the actual bug is in your code.

DAVID: Yeah. So, here's my strong opinion. Please, please, please consider this. If you've never done this, do it as an experiment and, like, do it for a week. Do it every time. And that is, whenever you run RSpec, put dash dash format equals documentation. Or, if you're, you know, quick and in a hurry, dash F Space D. This turns off the dots. You don't get green dots, red Fs, and little yellow asterisks. What you get is this nested tree of text, and it gives you the object you're describing.

And if you've got a context of, like, you know, happy path versus unhappy path, it will put happy path on a line by itself indented underneath the object that you're describing. And then, when you're in the happy path, let's say we're going to describe the initialize merchant, you know, method, so you've got a describe, initialize merchant, well, that shows up as the third line indented twice underneath happy path.

So, you get this beautifully indented tree of text, text, text, text, text. And then, you finally get, like, you know, it should create it at timestamp, you know. So, it's like, it sets created at your RSpec declaration, right? You get to that line in the documentation, and that entire line of text turns green if it passes, or the entire line turns red if it fails. So, that's the first half of the experiment.

The nuance to this is run it in format documentation and read it as if you had never seen your code before or as if you were sitting down to train another intern on how your code works. And you really, really, really want them to be able to read that stuff on the screen and not have follow-up questions. So, have them just read your documentation and go, hmm, okay, seems legit, I got it.

Eddy, this then circles back to your point. If you take the time to type out correct clauses in your thing...and it's real simple, never type context without typing when after it. So, I don't write context happy path. Instead of saying context user is unauthorized, I will write context when the user is unauthorized or when user is unauthorized. And then we get to the it, which is, you know, it redirects to the login page. It displays a 404 error, you know, a 401 error.

Here's the thing: when you run this in FD, in format documentation, the last step is force every single spec to fail and watch it in the format documentation. You will end up seeing, describe this API context, you know, on the happy path when the user is authorized. It logs them into the homepage. You get this lovely little documentation thing. And you're like, okay, yeah, logically, that follows all the way down. I got it. Cool, no problem. It's green. Everything is good.

Now, you make it fail, and then you scroll down to the error message. And RSpec will take all of those nested snippets of text and it will chain them together into complete mishmash gobbledygook that nobody looks at unless you were using the grammar. If you've been writing context when and, you know, describe, you know, da da da, also with, you know, context with this thing going on or with this flag set, all of a sudden, that error message down at the bottom...and these are the ones where you get, like, we're no longer showing the blocks at test. We just got, like, an error and the line of code that it was on.

You get this chunk of red text that literally forms a sentence in English. On the happy path, when the user is authorized, and they log in, they get redirected to the homepage. And that whole thing is red, and it fails. Let's say we say the user's password, you know, doesn't match. We're going to assert that the text "Welcome, Eddy"...we're going to set this username with the name of Eddy. The text, "Welcome, Eddy," should be on the page.

So, you get this long, red sentence. You know, on the happy path, when the user is authorized, they log in, and they get redirected to home. That's right. It failed. Then the error message on line 273 of your spec file: expected to find welcome, Eddy, but that text was not on the page. Now, without opening the code or the spec, you know what was going on when it failed. And you know what the program did that was wrong.

And I promise you, if you do this, this weird, oddly specific thing will happen, which is four out of five times, you'll go, oh yeah, we commented out the user welcome thing. You haven't even opened the code, and you know where the bug is. You know what happened. And that is this weird, oddly specific, delightfully pleasing thing that happens if you write context when, you know, describe thing context when it does a thing. And then, you run the format documentation and make sure it makes a sentence. It's lovely.

And then, anytime you have an expect, expect 42 to equal 43, and then see what exactly is the error message. Because you'll end up with somebody else's code, and they'll be like, "Uh, yeah, we tried to log in, but we couldn't. Oh, and the error messages on line 229 expected 42, but it wasn't." What? This helps me not at all. Well, it does help you. It's going to give you the line of code in, you know, what file and exactly the line of code. But now you have to open that file, and you have to go to that line of code.

Whereas if you get the error message expected to find "Welcome, Eddy" on the page but I didn't, you know exactly what's going on. And much of the time, you know exactly what you just did a few minutes ago. Like, oh yeah, we forgot to turn this back on. It's so delightful when that happens when you can literally debug your code without opening it. And a good spec suite will give you that.

EDDY: Yeah. I just kind of wish, like, the error messaging was a lot better. Like, this is sort of, like, going off the top of my head where it's sort of, like, class double [inaudible 30:10] expected arguments.

DAVID: Unexpected method, blah, yeah.

EDDY: Yeah. Blah, blah, blah. I'm like, that doesn't really make sense, right?

DAVID: Force your expectations to fail and run them and see how they fail, and then reconstruct your context. If it's really oblique...we had a meeting earlier where we were talking about whether or not we should display a link. And the code underneath it was, is this object not invalid? You can infer that we don't want to show this thing if the object is invalid. But I didn't realize in the API that if the object is valid, we absolutely should show it. There's no other conditions to check, right?

So, if you get a red dot and it said, "Expected true got false," oh, that doesn't help. Does it? Right? So, if we then turn around and say, when the link is valid, it should show, then when that fails, you can say, hey, I expected the progress bar to show the link, and it didn't. Or the, you know, I expected it to be show link or, you know, to have the show link enabled, and it wasn't. Now, that error message takes you exactly where it was.

There was another developer on the call that offhandedly said, "Why don't we just put, you know, not invalid in the code that calls it?" Show link is a conversation I want to have with a progress bar. Invalid or valid is a conversation I want to have with the model underneath it. And I don't want to have to remember that valid invalid implies show link, not show link, and that nothing else is part of that implication, right? I don't want to keep track of that.

So, like, what you're saying, Eddy, we don't want to say, on line 73, this code failed, expected false got true. We want to say, on the happy path, when this link is valid, when I ask the progress bar, should I show this link? It should say, "Yeah, yeah, you should show this link." And then, you get the error message: expected false got true, or expected progress bar to be show link, which is a little bit awkward English.

But, I mean, we expected that you know, to have the show link status be enabled, but it wasn't. And you're like, oh, okay, yeah, now I know exactly where we are. I know what conversation we're having with what object, and I know what the semantic meaning of that conversation is. Does that make sense, Eddy? I just realized you've now said twice this is hard to understand. And I'm like, oh, no, no, no, [inaudible 32:22]. And I just realized I'm just throwing words at you. Is this helping at all?

EDDY: Yeah, yeah, yeah.

DAVID: You can say no. The listeners will laugh.

EDDY: [laughs]

MIKE: You know, Dave is saying use RSpec differently, and the errors will make more sense. Distill all that down. And I think that there's a lot of truth to that. It's easy to map, and I think it's kind of instinctive (I think I did it when I first used RSpec.) to map frameworks that are, like, Test::Unit style that are run assertion, run assertion, run assertion to RSpec, which has a different paradigm.

RSpec is designed to be thought of as a documentation generator as much as a testing framework. It says [inaudible 32:59] the same thing. And if you use it that way, you will have a different experience, that you will be much less likely to have those. This error makes no sense because you're looking at as documentation. And if it's formatted differently, then you don't just get a bunch of green dots with one red dot and a cryptic message. Instead, you get this documentation, and you get the line in the documentation that doesn't work.

And you're like, oh, the documentation doesn't apply here, and that gives me something. It may not be perfect. And a lot of times, the errors are just, you know, Ruby errors. So, they're language errors rather than RSpec errors. And then you have to make, like, oh, so, I have to figure out what's going on with my tests. But you at least have something to hang on to. I do think that changing that mindset is a big part of how you make those error messages less cryptic, is you think about the problem differently.

DAVID: Interestingly enough, that is the reason...the thing that we just talked through, the error messages in particular, is the reason that I won't ever use the subject method in RSpec if I can avoid it. And I definitely...if I use it, I will not write it do without text or context do without text. And the reason why is because RSpec will almost obfuscate the error message. Like, you literally will get, example, 3 of 5 of 1 expected 17 got 15. Good luck.

Now, not only do you have to track down where the code went wrong or where the spec went wrong, but you have to start winding up through the top of the file to figure out what did the spec set up before [inaudible 34:26]? Yeah, just type. Type some text on the context and the it.

I realize what I described sounds really hairy. Like, I've shown a couple of people, and they're, like, oh, this is too much of a headache. Context when it does, and run it, and look at it, and make it fail, and read the sentence. It will take you less than half an hour to completely internalize it. 5 or 10 times through the loop, and you'll have it. It's opaque at the beginning, but it's a skill that you can master very, very quickly. It'll click very quickly.

MIKE: And, again, on the branches, what else would you write? If you're describing when some set of conditions apply, well, you probably ought to have some text that says when some set of conditions apply. It's just the natural...RSpec is opinionated without necessarily saying so. Think about your branch of code. What branch of code applies when these conditions apply? Or maybe if or, you know, it's conditional. It begins with a word that suggests conditionality that, again, maps to the way that you're structuring it.

SPEAKER: Speaking to the RSpec, errors aren't top tier; I was working on something the other day. And essentially, some of it just consisted of adding a new field to just a hash that was returned by some function call. And I was looking at the RSpec because it just kept failing. And I was like, what's going on? Like, what is the reason for this?

And it said that it failed because all the decimal values that are returned were returned in scientific notation instead of, like, just regular decimal notation. And I was like, why is this happening? This doesn't make any sense. Turns out it was just because the test file wasn't accounting for that added field to the hash. So, it was like, it didn't tell me exactly what was wrong. And it actually told me something wrong that was wrong.

MIKE: But you saw it said, "These hashes don't match," right? And you looked at it and said, over on the left side, these are a fixed number of decimal points. And over on the right side, they're not. And so, they were things that RSpec could say were equal, but when it displayed them out, they weren't quite equal. And so, you were looking, oh, these aren't equal. Where's the problem? Because, you know, it kind of obscured it because you couldn't tell where the mismatch was.

SPEAKER: Exactly, yeah. It was a bit strange at first, but having a very written-out English sentence would have made that about ten times easier to recognize; oh, this is the problem. That's why this is going wrong.

DAVID: There's a subversion or an extension of this. It's interesting because, like, when we talk about RSpec exception errors aren't top tier, I'm having this instinctive reaction. And I'm trying not to, like, am I trying to defend my baby here, you know, out of just loyalty? And I realize, no. No, it's because that context when and the it, you know, describe noun, describe when, you know, it verb, da da da...RSpec, for me, is top shelf. It is the best error message. But I realize now it's because I write them that way.

I've tried to port that to Minitest and Test::Unit, and you can kind of, but it's a lot more work. It doesn't flow as naturally. And that might just be a translation and familiarity thing for me because I'm familiar with RSpec. But I 100% agree that hashes do not match, especially if it says expected, and then you get, like, two and a half lines of text with a dot dot dot in the middle of it because RSpec finally just said, I'm just going to have to give you an ellipsis. But I actually got two and a half lines of text with a dot dot dot in the middle, right?

You can get RSpec to give you just as much information and not waste as much screen real estate by just typing RSpec or or echo; it didn't work. And that will just suppress all the output from RSpec. And if it fails, there's a mistake in there somewhere. Good luck. Have fun, right? And hashes [inaudible 37:58] math is terrible. It really is terrible.

EDDY: Well, that's why I started to adopt hash including versus double splat, right?

DAVID: Yeah. So, there's a subtle trick also that you can do. You can write this; I say don't. You can say we're going to describe this hash, and all the fields should match. And you can write out; it should match. And then, inside the it block, you write, you know, for key value in expected, actual subkey should equal, you know, or expect the actual key to be the expected key, right?

And that's great, except that you're going to end up with an error message that says, hey, on line 27 of this thing, which is inside a for loop, you have no idea which index you were on when the for loop failed. Because that thing did 150 expects, and you have no idea which one failed, right? And you're going to get an error message that said something like, "Expected blank string, got nil." Oh, good luck. Which field out of that was it, right?

If it was expected, 123 Fake Street got nil; okay, I've got a street address that's missing, right? But it's when you get something that's, like, it could be any of them, right? Expected, you know, AAA got BBB. Oh great, where's my As and Bs? You know, what does that mean, right? If you're going to play that game, take that hack and pull it.

By the way, this is where the fact that RSpec is compiling classes becomes like black magic awesome. Because you can take that hash outside, you can't use a let, but you actually have to define it right there in the Ruby code of key value, key value, key value. But you do it in the source code outside of the let, or the describe, or outside of the it or the describe, and then you say a key value. And inside that key value, that each loop, you say it, and then in the text you say, should have field, you know, and then the field name key equal to, you know, field value. Or, you know, field key should match, right?

What happens when RSpec processes that file it hits the loop, and it generates. If there's 150 key values in that hash, it generates 150 it blocks, expectations. And because you put the variable in the it description text, now when it fails, that is the text that RSpec will give you in the failure message. You'll actually get a thing that kicks out and says, "Hey, address line two does not match, expected a blank string, got nil." Oh yeah, okay, yeah, we weren't reading in the department or the suite number off of the address. Oh yeah, look at that. We don't even have that column on the model. That's why it's nil, right?

You can make RSpec lead you by the nose and point you, right? You know, hold your nose right up to the air and say, this is it. It's right here, and this is why it happened. But you do have to make RSpec do it. You have to learn, again, like, context when, context when. If you take anything from this, put when whenever you do context, and then describe a condition when on the happy path. It really does help.

MIKE: If I make one other suggestion --

DAVID: Sure.

MIKE: Take this cave metaphor a little further. I love what you just said about making sure that every single expectation actually, you know, on the it block or on the context block has documentation. And one other place that people tend to struggle with in Rspec is how to do the setup. How do I set up my before or my let? And this is maybe a place where there's less clarity. It doesn't force you as much.

But I have found...and it sounds like my thinking is similar to yours. You say you put the unhappy branch first. Well, I would say that's correct. If you're exploring a cave and you're looking for the way out of the cave, or you're going through a maze, well, you're definitely going to go through the non-solution first, right? [laughs]

DAVID: Oh, that's interesting. Finish your thought. I do it exactly the opposite. I put the happy path first. But carry on.

MIKE: Well, and I think that there's a very compelling reason, an extremely compelling reason to put the non-happy path first and have the happy path at the bottom, and it's this if you do your setup for the happy path, so your let blocks, you know, your before each, whatever it is to do your setup, your mocking, your stubbing; if you set that up for the happy path and then you begin by testing your unhappy path, then what ends up happening is every branch of your unhappy path you're only saying what's wrong if the authentication fails.

And then, you're overriding one piece of your setup. It allows every single context to have just a couple of things that's going to say, "This is the thing that's wrong." And you're going to see an it block that says, "And it doesn't work." And it becomes trivially obvious to read through your spec and say, oh, this is the thing that's wrong here. This is the thing that's wrong here. This is the wrong thing; it's here. Because at every branch, the only thing that you're overriding is the one piece of your setup that is broken at that part of the branch.

And then, when you finally get down to the deepest level, you've reached the bottom of the cave. And hey, there's the door out. There's no setup at all because you've exhausted all of the bad conditions, and all that's left is, hey, the setup works.

DAVID: Oh, that's neat. You were halfway through that, and I was making the meme of the stick figure guy who's going "[vocalization], you know, I've got a reply for you." And then, halfway through, he's like, "Oh," just kind of deflates a little bit. And he's like, "Hmm, yeah, you're right."

I'll extend it one more step, which is that if you start with a working thing, you are blacklisting your errors. Instead of whitelisting one functioning thing, it's easy to go through and break one thing, one thing, one thing, one thing. Anything you forget to break will not get tested, and the object will pass. If you start off with a blank object and only populate in every single test, that can get exhausting if you're populating all the fields, you know, unrelated fields each time through. But there's some value in that, right? Sometimes you're like, I want this object to kick and scream and just refuse to be valid until the last possible minute. Yeah, I like that.

MIKE: I imagine you could thread through this in either direction. You know, some people work their mazes backwards.

DAVID: There is a left-hand rule as well as the right-hand. It works exactly how you think. Yep.

MIKE: And the key part there is that this is what's different in this branch. Here is everything. You want to make it really obvious this is the part that's different, and this is what happens. And you want it to be designed such that you have to go through all the branches to get to the end.

DAVID: I kind of like that. From, like, a documentation, like, teaching a maintainer how this works, I kind of like having the happy path be first, from a, you know, like, this is how it's supposed to work. Now, let's talk about all the ways it can go wrong. But I like that idea as well of, like, yeah, this thing is wrong. It will not be right until everything is right. So, I think that's totally valid. It makes sense.

MIKE: Right. I think that either way, starting with the wrong or starting with the right could work that the critical part is that you follow a direction like that, right? And this is another thing that people have problems with with RSpec. You have to go digging around and looking for what the context is [inaudible 44:30]. The only thing that you should see in your context is what's different. You could say that, well, I just can do all of my setup. Here's everything that's right, and here's the one thing that's different. But I think that that can actually be problematic because it's noisy. I want as little noise as possible.

DAVID: Yeah. I had it driven into me recently that there's no such thing as a solution; there's only a trade-off. And you have to decide what side of the trade-off you value. You might come into a thing where you know what? The left-hand rule is actually more important. I want to start with the wrong end of this maze and work backwards. And on another day, you might walk in and go no, no, no, no, we need the happy path. And then, we kind of don't care about the unhappy paths, so they're secondary. Let's just document the right way to do it, right? And so, it can go either way.

I have one last thought on the right-hand versus left-hand hand. If you're listening to this and you don't know who Steve Mould is, M-O-U-L-D, he's a British mathematician, scientist, YouTuber. He's really gotten on this kick lately of building physical models of logical ideas. And what he discovered is that you can use water to solve a maze. By pouring water into a maze, it will always solve the maze, which is really, really neat.

The reason I think it was lovely—and this is the right hand versus left hand—any system you divide into binary pieces will, when you get to the other side, fall into two separate halves. They might not necessarily be equal halves, but there will always be exactly two halves. And Steve discovered this in his...if you go watch Can Water Solve a Maze, he was laser cutting acrylic mazes, clear plastic so that he could pour colored water for the camera, for YouTube, right?

And he said, "This weird thing started happening. I started cutting out these mazes. And when the laser got to the end of the maze, half of the maze fell off of the cutter. And the other half, it was exactly two pieces, the left-hand side of the maze and the right-hand side of the maze." Whichever rule you follow, if you put your hand on the right-hand side of the maze, you will not touch the left-hand side of the maze ever. And vice versa, if you start on the left, you'll never touch the right half of the maze. It will divide into two exactly separate halves. I thought that was super cool.

MIKE: That is cool.

DAVID: Folks, this was lovely. Thank you for coming out on the podcast. Hailey, I appreciate your comments. Dom, Eddy: love having you guys chat on the call. This was a lot of fun.