Favorite Dev Tools and Tricks

About this Episode

We're calling out the most valuable development tools and tricks.

Transcript:

MIKE: Welcome to another episode of our Acima Development Podcast. I'm Mike. I'll be hosting today. And today, we're going to be talking about development tools and tricks. Before I do that, I'd like to go around and have everybody briefly introduce themselves. Mark, do you want to introduce yourself?

MARK: Hello, my name is Mark. I'm an intern for the summer.

MIKE: Thank you. Jason.

JASON: My name is Jason. I've been developing for about six months now. I started as a developer here in Acima.

MIKE: Great, thank you. Ramses.

RAMSES: Hello, hello. I have also been a software developer now with Acima for about six months.

MIKE: Great. And Kyle.

KYLE: Hey. Kyle, DevOps Engineer, been doing that here at Acima for about five years.

MIKE: So now that we've introduced ourselves, I wanted to start with not quite a story; it's an example. I'm interested in pedagogy, in teaching, in the science of teaching, and I've read quite a bit about math teaching in particular. There's a popular technique that has become more popular in the most recent few years to help children develop the appropriate mindset, and it works for adults as well, develop an appropriate mindset for math because, a lot of times, we're taught mindsets that are not very helpful.

Learning math by rote, in particular, tends to lead to difficulty solving novel problems. You run into a new problem like "Oh, now what do I do?" And professional mathematicians approach math very differently than how it's often taught in schools. For them, math is art and creativity, and looking for interesting new ways to approach problems, where a lot of people who are taught math in school are taught very differently, where here's your problem; here's the technique for solving the problem. And that's the one way to do it.

So in order to develop a better mindset, a more flexible mindset for teaching, they do what they call number talks or math talks, where you present a math problem that you can solve in your head with some challenge. How would you add 23+84 in your head? And you ask a class of 30 kids this, and you get like 32 answers. [laughs] Everybody does it a little bit different. And I may be exaggerating a little bit because you'll tend to have common themes.

But it's remarkable how people will come up with very different ways of doing it; some people will add the tens first and then the ones. Some people will try to get to easy numbers like multiples of 25, or multiples of 5, or multiples of 10 and then balance out the numbers accordingly. Some people might go to 100, you know, they work around boundaries like that. There are lots of different ways that people can approach problems, a surprising number of ways people can approach those kinds of problems.

And once you hear all the other different ways, you're like, wow, I thought there was only one way of doing it, and I thought that it was just my way. And it turns out there are many different ways of solving the problem. It's worth looking up if you're ever interested in thinking about it. It's fun to try it with a group of friends. Just come up with a problem like that. Just do some addition or some multiplication that you can do in your head with a bit of challenge, and ask other people how they do it. And you'll likely be surprised how many different ways people do it.

I used this introduction to come into software development tools because, coming into the industry, you might think there's one way to do this. But our industry is a creative industry. Our job is not to follow a set of rules but to solve problems. I'll repeat that repeatedly, and I've probably mentioned it other times I host the podcast because there's a misconception sometimes that our job is writing code, which is not really true. Our job is to solve problems.

And there are a lot of ways to solve the problem. And you may have people who approach the problem in very different ways and come up with a great solution through very different techniques. And that doesn't mean that one is right and one is wrong. So there's some real value in seeing a variety of ways to find out what works for you. There's not necessarily a right answer. Sometimes there may be solutions that are not very good, and it's good to recognize those.

But, in general, there's a lot of gray area there. There's a lot of flexibility and areas where there's not really a right or wrong answer, just a variety of answers. And having a whole toolbox with the tools rather than just one hammer is going to give you a lot of power to do things differently and likely do things better because you can reach for a variety of tools and find the right one for the right place. And you may find that your personal preferences lead to different approaches from what other people do.

I'm going to lead by saying that I am very much a minimalist. I used to use big IDE when I was a Java developer years ago. That's pretty common in the Java world. And I did a lot of remote server administration for a while. I started using command-line editors and just stuck. So I'm a Vim user, primarily, and have been for a number of years, which is just a command-line editor. I never leave the terminal. I rarely leave Vim because I can execute commands from right inside Vim.

I don't have a lot of plugins and just use the standard Linux or Unix toolset that is available from the command line to do my searching, and I find it extremely effective. I'm able to navigate around and find what I need very quickly. Others use a big toolset and are able to also navigate around files and find things very quickly. I'm not going to say one is better than the other because I think they both have merits.

So I'm going to start with just saying I tend to be a minimalist. I'm curious where other people are at because other people have their own tools that are very useful. I will say that knowing at least one command-line editor...and the two big capable command-line editors are generally Vim and Emacs. It's a surprising superpower. It takes some doing to learn. But those tools allow you to accomplish a lot with a little. And even if you don't get really good at them, it's nice to have those in your tool belt so that you can accomplish something.

So if I were going to start by saying a very useful tool, I would recommend learning a command-line editor. And if you want an easy introduction, there are very easy command-line editors you can start with, like Nano, which is a simple editor that once you get into, you'll be able to see how to get out, unlike Vim. [laughs] And you can get started that way. I think there's a lot of value in using those command-line tools.

I would like to talk more about the incredible value of the command-line tools that come from Unix, particularly grep. Grep is the open version of this that's widely used on Linux that I think are invaluable for myself in my tools. But I don't want to talk the whole time. So I wanted to lead with that, talk about the importance of a variety of tools and start my take on useful tools by saying to anybody that command-line tools are, in general, incredibly valuable.

And I'll talk a bit more about that in a minute. But I'd like to hear what other people have to say about what, you know if you were to just...if somebody was to ask you, "What is most valuable to you?" what would you say is the most valuable tool?

KYLE: I'll have to agree with you on the Vim subject. And I have been around the Emacs people as well. And I feel like that requires a different set of joints in your finger to use that tool.

MIKE: [laughs]

KYLE: But I come from the other side of the fence as well, though, because starting out in QA, a lot of the tooling that I was introduced to that made my job easier and flow well was all GUI-related and Git from a UI and stuff like that. The transition to using something like Vim, I wouldn't say I'm any kind of expert. But I feel like it's made my job so much easier in the sense that there is native tooling, like you were saying, grep, and everything in Linux and Unix that you're able to do that's even cross-platform.

A lot of what you're going to find on Linux is available on a macOS or even in the later versions of Windows. Now you have that ability as well, either through PowerShell or through the Linux Subsystem that they have now. It's just I do feel like my productivity has increased. And the scripting and the tooling that you're able to use, even with just a Bash application, you can just do a lot that you can still do with a UI as long as you're really familiar with the tool.

MIKE: A lot of things you can. There are some things that are actually remarkably difficult, if not impossible, to do from a UI. I don't want to get to the punch line because [laughs] I want to hear what a couple of people have to say. But I'd like to grab back onto that thread because I am with you completely that it is useful, and you can do it from a UI if you're familiar with it. But there are some things that I think you can't in most UIs to accomplish. Again, I'm going to come back to that. I'm curious. Jason, I think you were going to say something.

JASON: Yeah, I think it was drifting a little further off-topic from a specific IDE. But a really useful tool that I had when I came on, Johnny actually pointed out. And basically, he showed me where I can customize methods anywhere. I run Pry or anywhere I run an IRB console. So just knowing where you can add those custom methods is really a time saver and kind of lends itself to what you're saying, you know, running things from a command-line editor or a GUI.

Right now, I'm using RubyMine. It seems to be popular with a lot of the devs and has a lot of useful tools, and being newer, that is helpful. There are a lot of helpful hints there. But yeah, being able to modify your pryrc file or irbrc file to add some custom methods to help you save time has been big for me.

MIKE: Thank you. And anybody listening who's not a Rubyist, Pry is a popular Ruby debugger. You can use it on the command line.

ZACK: So one of the big things that have always gotten me throughout my development career is merge conflicts. And the tool that I found that I've just hooked into the terminal so I can just run git merge tool and pops up is Meld. I don't know if any of you guys have ever used that before. It kind of does pop up like a UI for you with some easy arrows to just kind of push and create one file with the right edits to it. And it's been kind of a lifesaver as far as merge conflicts have gone.

MIKE: Merge conflicts are the bane of development. [laughs] Anything you can find that is useful for those is good. We could have a whole episode on Git and ways to deal with merge conflicts and avoid them before they ever happen. Thank you. Anybody else who wants to share a tool that just really is the best one that they use?

MARK: For me, it'd be VS Code open-source code editor that you can pretty much do anything you want in. And the only problem I would give with it is that there are sometimes troubles with its terminal. But besides that, I find it's a great source for editing code as well as the ability to customize the interface and so forth and style it as you like.

MIKE: I feel like VS Code has kind of taken over the world. [laughter] It seems to have become the editor of choice for a lot of people

MARK: Well, more and more people are pushing for open-source products. I think Germany swapped all their government computers to Linux. Or was it that they put their whole school system on Nextcloud?

MIKE: I know there have been several experiments in Europe like that. I haven't followed it really closely. Definitely, you know, open source is helpful. And they've got a big corporate backer for VS Code as well, so it keeps it going. And we've also got some interesting integrations now with Copilot, some very interesting things with that IDE. They're very cool. So there's a lot of innovative stuff going on with VS Code. And it seems to be where a lot of the energy of the industry is.

KYLE: Just wanted to pop in and say something about VS Code just because that is another tool that we heavily use with DevOps because, once again, it integrates with everything under the kitchen sink. Just...and, Mark, I don't know if you've tried it or not, but you can basically use any terminal with VS Code. You just have to go into the settings and pick which one you're wanting to use if that helps you out at all.

MARK: I'm aware. It's just there are some times where I've run into a permissions issue where it'll say, "The terminal within VS Code has less permissions," then just pulling up the terminal app.

KYLE: Ah, okay, yeah. It can be limiting at times.

MIKE: Well, with that segue going back to terminal, I want to revisit this idea of command-line tooling versus user interface. There is a philosophy embraced by the Unix operating system. Nobody...well, I say nobody...the original Unix operating system birthed many children. And while pure Unix is not that widespread anymore, it has many of these children. I believe that macOS is formally certified as a Unix and meets the requirements for Unix, even though it has a very different underlying system.

And Linux is Unix-ish in its approach, even if it was built from scratch. And it's taken over the world. Android runs on a Linux base. I'm actually not that familiar with iOS, and it wouldn't surprise me if they used something similar. Because most of the devices out there in the world run some Unix-inspired operating system, and I think there's a reason for that. There's a Unix philosophy that says the tools should be small, single-purpose, and composable, all of which are important.

I say small, meaning that they don't do all of the things, although some of the...like an editor, maybe does a lot of things. Emacs, there's very little it can't do. [laughs] Some people see it as unwieldy, but it's a very powerful tool. But most of the tools they use are single-purpose. You might have a tool, for example, that counts characters or a tool that just runs regular expressions to search for text within a string.

And the single-purpose tools are composable in that Unix allows you to create what they call a pipe, which sends the output from one command as the input to the next command, allowing you to chain commands together and compose an arbitrary number. And you can put 50 of them together. There's no real preset limit. You can take the output from one command and put it as the input into another. And, okay, that sounds simple enough, but it is radically more powerful than other approaches, and I'd argue that emphatically.

Because if you've got a graphical tool that lets you do a thing, maybe it lets you do a sophisticated thing, you can get whatever you can get from that one tool. But instead, if you have a simple tool, instead of doing some very sophisticated thing, it does one thing well. And it allows you to compose it with other things that would accomplish the same thing.

You can almost always do what you're able to do with that graphical tool but then take the output of it and send it to another tool. So you can take your sum that you got from the first tool and send it into the next thing to do some math on, and then send it to the next tool to print it out to a file, write it to a file, and then you can even send that file as an email. You can chain these tools together to come up with a very sophisticated workflow.

A lot of times, just in development, you want to do simple things. What if I want to look for all the files that have a particular extension and, for each of those, search for a particular line of text, and then compile all of those files into a list and open them up in my editor? I do things like that all the time using my command-line tools. It is very useful. And that's something that's actually very challenging to do using a graphical tool and may not be even possible in most of the tools that you use unless you've learned how to use some obscure tricks in the edges of it.

And that's why I say that these command-line tools are, I think, underappreciated, and the most important tool, I think, that is worth developing and getting in your tool belt as a developer because they are much more powerful than most other tools available and faster as well. And, Kyle, you talked a little about the power of command-line tools. Has that been consistent with your experience?

KYLE: Yeah, that's along the lines of what I've experienced, too. I was actually sitting here thinking, as you were talking, there are CLIs for everything almost. And, I mean, if you're looking at a...not to bash on any particular company, but you go in, and you'll see a UI, and you'll familiarize yourself with the UI. And then four months later, if we're looking at long-term or whatever, you go in there, and it's like, oh, we've got a new look. And you're like, well, that's great, but where's this setting, or where's this feature that I was looking for, right?

MIKE: Right.

KYLE: And with a CLI, I've found that doesn't change; that's constant. They may add features, but they don't really move them around. And I feel like that's another power of the command line, if you want to say, is each of these companies are making CLIs that they will do the same thing, call the same APIs is what, you know, either their application or web interface will do. And you're able to do a lot of the same stuff easier and faster than you might be able to even find it in the UI.

MIKE: I agree 100%. Just anybody listening who's not familiar with the lingo, when we say CLI, we're talking about a command-line interface and using something from the command line. I'd ask that further, Kyle, from the DevOps perspective, which would you rather have: configuration files that you can manage through Git or, I guess, there are other source control tools, but nobody...I'm not going to say nobody uses them [laughs], but Git has kind of dominated source control. Would you rather have configuration files and scripts managed in source control or a pipeline managed in a graphical user interface?

KYLE: I avoid a graphical interface as often as I can. It's all about having configuration management.

MIKE: Yeah, there's a reason that DevOps has become central to, I think, most effective software companies. They treat configuration as code and use the same powerful tools that we use in development to great effect.

KYLE: It's just one of those situations where, I mean, here at Acima, we've got...what is it? Five or six environments. And so, I can either go in and configure each environment through the UI, or I can configure one environment in configuration and apply it to every environment with a quick script. And it's which one would you rather do? Which one's going to save the company money and make you look better in the long run?

MIKE: [laughs] Absolutely. We touched on something there that we didn't dig into. We've been talking about command-line tools and their great utility. We talked a little about Linux. But there's another incredibly powerful tool written by the same guy, Linus Torvalds, who gave us Linux. Linus Torvalds started Linux [laughs] but has personally not, by any stretch of the imagination, written most of it. His gift to the community was planting that seed and starting that community, which together builds a remarkable tool.

He also started another tool that has become fundamental to, I think, most development companies out there which is Git. It's kind of a silly name, incredibly useful tool. [laughs] And maybe it'll be superseded by some other tool in the future, but for now, I think Git sort of reigns supreme. You can use graphical interfaces for Git. I actually don't, like, ever [laughs] because its command-line interface is very powerful. But graphical interface, there's nothing wrong with them; they work great.

And having that powerful version control system is, I would say, a vital tool for effective development. Those of us who have used, I would say, inferior or simpler tools in the past know the treasure of everything that Git is. And Git is actually, at its core, a pretty simple idea, which is instead of having a central repository that has a single point of failure, everybody keeps everything.

You keep a history of all of the changes that have ever happened to your project. So you have basically a stream of events that are signed so that you can verify them, going back all the way through the history of a project back to the beginning. And everybody keeps that. And by having it distributed, it's very hard to lose very much that's important.

Maybe you've been working on a project for a couple of days and kept it on your own machine and didn't share it out with the community, and maybe you'd lose that. You don't need to. You can push up your changes hourly, and then you're unlikely to ever lose very much, and certainly not the whole project. And Git manages that distributed system of multiple copies repository and manages to handle it very well. And in doing so, it has become a central tool to most development. And there are business models built on top of this open-source tool, such as GitHub. What would your lives be like without Git?

MARK: A big pain in the ass for managing.

MIKE: [laughs] Yeah.

JASON: A lot of re-dos.

MARK: We'd probably have something like daily merge meetings just to begin work together to merge all our changes on everyone's devices.

MIKE: Oh. I think that's exactly right. You'd do daily merge meetings, and you'd have to carefully coordinate. [laughs] Let's make sure that you never touch the same code as somebody else. And it would all have to be done by direct communication. You'd have to talk to somebody first to coordinate that. No easy way to handle conflicts when you both accidentally change the same file. Nightmare. [laughs]

If you're starting out at development, start using a version control system. It will serve you well. It will save you pain, as Mark said, and re-dos, as Jason said, more times than you will ever count. It's just a remarkable tool. And there are some other systems out there. Git is the most popular and extremely useful. There are a couple; like I said, there are a couple others. Mercurial is used somewhat. It has its own following.

JASON: One added benefit to that is you can have kind of this fearless sense of I can experiment, do whatever I want. I'll just create a new branch and experiment and see if it works.

MIKE: And it changes your mindset, doesn't it?

JASON: Absolutely, yeah. Also, this discussion is kind of interesting because I never really thought of source control giving you that liberty. I'm not afraid to touch any file. I can do anything that I need to so long as I'm not modifying something else, like a database that could set me back, but even that, you can reset to a better state.

MIKE: I love where you went there. It gives you freedom to experiment. Freedom to experiment is one of the most important things for being able to be effective as well. It's closely related to the idea of psychological safety, which research has shown to be the central characteristic of effective teams and effective developers. If you can feel free to experiment without harsh consequences, then you're going to, and trying new things is how you get things done. I think we could talk about this for some time.

I saw a wonderful presentation at a conference once talking about luck. And I'm going to summarize the presentation, and you can go and look it up yourself. But there was some research done about what makes people lucky. They asked people who of their friends was lucky. So they characterized a group of people who were considered lucky by others. And they figured out what was different between these lucky people and other people to actually research what makes a person lucky. And you can figure out what the difference is.

It turns out that the key differentiator between lucky people and people who are not so fortunate is that the lucky people try new things. And when I heard that, I maybe paused; oh, really? I've thought about that a lot since. If I want to be lucky, I got to try new things. You're never going to have a different experience unless you're trying new things. Yes, you can have some bad experiences too, [laughs] but you're never going to get something that will take you to the next level, have that moment of serendipity unless you are putting yourself in a position where you can. Git allows you to be lucky.

KYLE: Just a side thought on this as we're talking. I almost feel like Git, or a tool like Git, is partially why some of us can be employed I feel like. Because I just feel like if you didn't have a tool like that, large development teams would be a burden more than anything. You'd have small, concise teams that don't, you know, one dude is working on something, and then the other person is working on something else. And they can quickly merge things.

And we wouldn't be able to have 7, 10 developers, whatever, working on the same service because they would be having those conflicts, and it would be just a managing nightmare. So I think part of this is that it gives us that parallelism to allow more developers to be employed to deliver faster and everything.

MIKE: Well-spoken. Going back again to that capacity, it'd be utterly unmanageable without the tool.

We've talked a lot about the power of the command line and the power of version control. I think I mentioned one command-line tool that, for me, is perhaps my favorite, the one that I use all of the time and I think is perhaps under-appreciated, which is grep. I mentioned it previously. Grep is like a utility knife for search. [laughs] It has all the various different tools out there that you might need to use.

It's a fairly simple tool. It searches using simple expressions, generally uses regular expressions. You can specify how much it leans toward regular expression versus just simple text search. Regular expressions are a little mini-language for search that are used widely throughout the software industry, including in tools like grep.

And using that tool allowing you to search on the command line is amazing because if I want to search for, say, a string within a set of files, I can use that tool, and maybe I'll get back 10,000 results, way more than I wanted. But then I can pipe that into another usage of grep that reverse filters for some string that's common that I don't actually want in my search. And maybe I'll chain three or four of those together because I'll run it and see my output, like, oh, wait, I've got this other string that is always there when I'm actually looking for just a substring of that. And I'll just keep adding to my chain.

And maybe eventually, I'll get to a chain that's 10-long where I'll filter things out that I don't want, starting with something that I do want. And maybe at the end, I add something else that I want, and I'll get back a small set that gives me a list of files. And that chaining is remarkably helpful. And some IDEs, I think, support this idea of chaining.

And I think internalizing that idea that I don't have to do it all at once. I can do this in incremental pieces and chain them together to get what I want is remarkably powerful. That is very powerful for grep. But it's also powerful in a lot of other contexts. Thinking about this pipeline of chain commands that progressively filter and transform what you started with leads to an incredibly powerful tool. Any other heavy grep users here?

JASON: I use grep specifically every day.

MIKE: It's a superpower, isn't it?

JASON: It's a time saver. It's nice. That's one of the things that it's kind of funny. I'm starting to see why people transition away from a GUI to command line, or at least now, whereas before, I didn't always have my terminal open. My schooling was in Java, and it wasn't super handy having the terminal open, at least not for me. I wasn't experienced enough. Definitely, now it's constantly open, and I'm, again, starting to see that transition because it's just so much faster. Maybe that's why Ramses could get so much done, superuser.

RAMSES: Yeah, I don't use any graphical interfaces; it's all Vim.

MIKE: It's just so much faster, isn't it? [laughs]

RAMSES: Yeah, incredibly. I've tried to use different graphical interfaces like RubyMine and VS Code, but they just slow me down.

MIKE: That's where it got to for me; I mean, I used to use an IDE all the time. And after using a console-based editor heavily, when I tried to go back, it was painful. [laughs] This is just so slow. There are very good uses of graphical IDEs, though, that can become quite effective. Again, there's more than one way to solve this problem. But people who go to the command line, I think, in general, get very fast. [laughs] It's worth doing. It's worth learning, even if you don't use it all the time.

We actually had a late joiner who spoke up once, Zack, who works on the data side. I'm very curious because we've got mostly traditional developers. But data has kind of taken over the world in the last few years. So I'd be really interested to hear, Zack, what you would consider some of the most useful tools. You talked about using tools for solving merge conflicts. I would guess that you have several other tools that you might see as vital. If you don't mind, Zack, what you mention might be useful tools from the data side that the rest of us might not be familiar with.

ZACK: I use mainly the JetBrains suite as far as IDEs and stuff go, specifically data-side, DataGrip to be able to just access all the databases, look at data like that. I don't necessarily really have tools that go through the data. It's all just queries that I have to write to find outliers and stuff like that. So DataGrip is just extremely helpful.

MIKE: That's interesting. So I'm assuming that SQL is your best friend.

ZACK: Yeah. Yeah, yeah. If you take a look at how our scripts function, we use Python, but it's really just like a gateway to SQL. Most of our stuff is just random straight SQL.

MIKE: SQL is SQL, S-Q-L. I'll use whichever one the person I'm talking with uses, [laughs] but it means the same thing. It's not necessarily the latest, greatest, most trendy language, and it's probably got some rough edges even. But what an incredibly powerful tool.

ZACK: Yeah. There are very few things, in my opinion, that Python or really any language can do that you can't just do in SQL as well, as it's extremely powerful for that data and never even leaves the warehouse, so you have no network latencies. You have none of that working against you. It's all just happening in the same place.

MIKE: I think that can hardly be overstated. But most operations, if you can keep them in the database rather than bring them to the client side, you will save yourself multiple orders of magnitude of time.

ZACK: Yeah. And there's still...with warehousing; it's typically a cluster of machines, whereas I believe maybe our Postgres instances are just one machine. So there's still a little bit of network latency that happens, even processing just within the warehouse. But it's so small because all of those machines are sitting right next to each other. But yeah, you can tell a massive difference if you were to run a query that pulls data out versus just run a query that creates a new table. It's night and day difference for sure.

MIKE: One thing I've noticed working with a data warehouse is that, as a general rule, all queries take a few seconds. You run a simple query to return a few records, and it takes a few seconds. You run some incredibly sophisticated query that does all kinds of complicated things, and it might run a few seconds. [laughs]

ZACK: It's kind of funny because those complicated queries that you're talking about will probably return faster for you than SELECT * from, just the way that a columnar database is geared towards a row-based database such as Postgres where the whole row is sitting right next to each other. But, again, in a data warehouse, it has to go search all over disk to go find all the different columns to combine them back together. So the most powerful tool [laughs] in my arsenal is the warehouse, very specific type of database.

MIKE: Yeah, I first started using a columnar database ten years ago or something, and it opens up a whole world of possibilities. When you just have your online transaction processing database, just your standard database, and you want to run some big, ugly query that you know is going to run slow, you pause. Can I really run this, or am I going to take down production? And the answer is, yeah, you probably shouldn't run it. And then your hands are tied. And what do you do?

And if you have a replica, then you run the query, and it works. But again, if you're running just a traditional row-based database, it might not ever come back. It might run for several minutes. You just run out of memory. It's not designed for that kind of use case. But when you have this other database that is configured differently and designed for running those big, ugly queries, it's not going to be very fast for, like you say, SELECT *, you know, return this one record. It's going to be slow. You never want to use that for your production database. But if you want to run a report, it's transformational.

ZACK: Yeah, it gets really, really powerful once you start separating compute and storage and all that stuff. And you can really have machines that are just specifically made for one purpose rather than just having one machine stood up that's really made to try to handle everything. Plus, just getting distributed computing going is a massive win when it comes to data.

MIKE: And those who haven't used it, it's hard to express the difference between waiting for 20 minutes for a query that times out and waiting 5 seconds then you get your results back. It allows a different cadence to your work. If you can do something 1,000 times a day versus 3, it's a different kind of work. It's fundamentally different in character. Sounds like you're going to say something, Zack.

ZACK: Yeah. My last job that I was at, when I started there, they were using a MySQL database as their warehouse. And it was exactly that. I would run a query, and I want to be outside of the range of expectation that it takes 45 minutes for that query to come back to me. And then I'm pretty sure they just had all their settings to negative one, or zero, or whatever, so there was no timeouts going on on those databases.

But they were also insanely easy to bring down. I had released code there, and for some reason, they were running a five-server leader configuration, which just didn't work at all. I did a code release there, probably my first one that just took down all five of those.

MIKE: Oh.

ZACK: Because you had to just write your SQL so perfectly to pull the data outside of those types of databases. And even then, like I said, you're waiting 45 minutes. And that's just the most inefficient way. I'm sitting there for 45 minutes, not able to continue developing my script because I'm just waiting on data to return to me.

MIKE: Going back to what Jason was saying before about the freedom to experiment, you have very limited freedom to experiment in that kind of environment. And that data warehouse gives you the power, the ability to experiment, again, takes the shackles off. Now you're free to go try stuff.

ZACK: Exactly, yeah.

MIKE: It's interesting that SQL and data warehouse came up as a vital tool. Do you have anything else that you'd bring up from the data side, or do you think you've covered the key items, Zack?

ZACK: I think that I've made the key items. I mean, you're familiar with Dave. He ran the show last week. He also develops inside of the terminal, like you guys were talking about. I would mention that I did end up inside of an IDE. I used to use VS Code. I ended up inside of an IDE, specifically JetBrains, because I started delving in Scala. That all has to compile at runtime. It was so much easier just to press the play button in an IDE than doing anything inside of a terminal.

So you guys were discussing how the terminal is extremely powerful and faster to use in that way, and I don't disagree like 90% of the time. But I do believe there's that 10% of the time where having [laughs] a UI tooling is just remarkable. Like, I dev Android apps, and I wouldn't want to do that without Android Studio spinning up an emulator for me to work on.

MIKE: That makes sense. And when I was a full-time Java developer, I used the JetBrains suite as well, IntelliJ, all the way. It's an amazing tool.

ZACK: It's funny; I really tried to use the terminal. And I was just getting these weird errors in Scala just because it wasn't compile...I was trying to not compile. I was trying to run a scripting language, just read it out and run it instead of compile it down to bytes and run it that way. And as soon as I jumped into IntelliJ and pressed that play button, it all just ran and worked but could not get it to work inside [laughs] of the terminal.

MIKE: It's not that that's an insurmountable problem but, like you say, the ease of use. I'd actually recommend that most new developers use an IDE, use an integrated development environment, a big graphical editor that lets you see things because it's a lower bar to entry. Why not use a powerful tool that makes it easier for you? But learn the command line too. Treat that as a goal. While you've done the easier thing, try using some of the command-line tools. Try experimenting with them here and there over time.

ZACK: I agree with that as well. I spend a lot of time in the command line. And I spent a lot of time early on through college working in networking and setting up headless machines that you just had to SSH into so there was no GUI that you could work with. And through that time, the three big ones that I would say if you know these, you can do just about anything would be Grep, as we discussed, sed, and awk.

MIKE: I use sed a lot. I need to use awk more. It's also a remarkably capable tool.

ZACK: Yeah. And that's one that I never really use a lot, either. [laughter] I never really hit cases where grep, and sed just didn't work for me.

MIKE: Exactly.

ZACK: And on the rare occasions I did, it was just like, I'd Google to figure out what command it is, and it was [laughter] typically awk.

MIKE: That's pretty much where I'm at. [laughs] Grep and sed encompass almost everything. And then awk, you know, that's what Google is for. [laughs]

We're reaching the end of our scheduled time, but to go back and summarize, we've talked about some powerful tools over on the data side, specifically, data warehouse using SQL or SQL, whichever you prefer, you want to call it. We've talked about command-line tools, particularly grep. And we talked about the value of version control system. We talked about Git and the value of IDEs. And the commercial product from the JetBrains suite of commercial products are really very good tools, as well as VS Code, which has become very widespread within the industry.

Hopefully, if you're listening, you've got some great ideas of things you might want to try. Is there anybody else who wants to put in a plug for a tool that we haven't covered or talk about some concepts we touched on, and you really want to dig in a little further?

KYLE: I realized when Zack was going over tooling that they use in data, I didn't really talk about some of the major ones we use in DevOps, which is I would throw out TerraForm, Kubernetes. Those tools specifically because...TerraForm is how we manage all of our clusters and all of our infrastructure out in AWS and Google and wherever that's managed.

ZACK: Just another one, real fast; Kyle reminded me when he brought up Kubernetes is we have started using a tool called Argo. It's a workflow management tool: also extremely important. In your normal software and engineering environment, you have typically something that happens that causes an event stream that runs through and kicks off whatever endpoints or whatever needs to be kicked off. A lot of times, with data, you just have to schedule that. So we use Crontab as well as Argo to schedule our jobs to run.

MIKE: Cool.

KYLE: Last plug, Docker. Docker has to be mentioned.

MIKE: Yes. [laughs]

JASON: The debate rages on.

MIKE: [laughs]

MARK: In Docker's favor.

[laughter]

MIKE: So I'll hopefully tone down some of that debate by saying that Docker is widely used in production and even on production deployments. In cloud deployments, having containerized code is just incredibly useful. For local development, I think there's a lot more debate as to whether you just want to run bare metal or whether you want to run containerized. Is that what you're touching on, Jason?

JASON: Yeah, just locally, obviously, it's resource-heavy, but I absolutely see the kind of the compartmentalization; like you said, the whole point is to run in containers. I can see where in a production environment, that's extremely useful. But yeah, there's been a lot of debate here at Acima over whether Docker is...if we should rely on that for local development, just the memory requirements, basically.

MIKE: You know, interestingly, we do rely on Docker, many of us, for local development, even if we're not thinking about it because we'll connect to cloud instances for some of our services, which are built using Docker.

ZACK: I was going to say my argument for running Docker locally, at least from what I've found, is it only uses the resources that are needed for that, which for my team is really easy because it's only using resources if we're running a script. As well as it's been extremely helpful to have a development environment that is pretty much a one-to-one to production. You don't get any of the sneaky, like, oh, it's failing in production, and I have no idea why it's going on. It will fail on your local machine if it's going to fail in production.

MIKE: It's wonderful to be able to compare.

KYLE: I think that is one benefit with Docker that is a little bit overlooked is we get this response from devs sometimes, you know when something is deployed, and it's...well, it worked on my machine. Well, that's great, but we can't put your machine in AWS. However, if you give us a Docker image, we can put that in AWS, and that's the same image that will be running on your machine as will be running on a machine out in AWS.

MIKE: That's a big deal, that consistency.

ZACK: If you look at toolings before Docker, it's not a new idea of trying to match local to production environments. Before that, there was Vagrant, which admittedly was maybe a little bit of a shit show; hard to manage but definitely been going for that for a long time. And I think Docker kind of hit the nail on the head with it.

MIKE: It's becoming remarkably successful and integral to a lot of what we do for a reason. It kind of takes the virtualization idea to its logical extreme. Let's strip out everything extraneous and have just what we need but everything that we need. All of your system libraries and your application that depend on them; we'll put that in a bundle and nothing else. It's a powerful, powerful way to approach deployment and even local development to get that consistency.

MARK: Docker has also been great for privacy and so forth. I know recently I discovered that you can run full-blown browsers within Docker and then access those browsers from your other browser. So I can have, say, 30 Firefox containers in Docker and have each container for a specific reason. So instead of creating profiles on my main computer for everything I want, I could have containerized Firefox applications for what websites I go to, which I believe is a great plus for security and not just development means.

MIKE: Yeah, if nothing else, you can segregate home from work. [laughs]

KYLE: I will say one thing that I've found developing in Docker is I tend to really kind of mess my machine up, and a local machine that requires a lot of debugging and figuring out what have you done and backtracking. The Docker image, use it, abuse it, throw it away, start up a new image.

JASON: I think that's a huge selling point for me going back to what we were talking about before with the freedom to experiment. I think that's pretty powerful.

MIKE: Well, it sounds like a good place to tie this up. There are a variety of tools that give you that power to experiment, and they've been coming up repeatedly. Docker and Git were mentioned particularly in that regard of the data warehouse. Anything that allows you to try something without breaking things is going to make your workflow much more powerful.

Thank you for joining us today. And, hopefully, we gave you some ideas for things that you can go use yourself. And I don't know if we'll be seeing you, but you'll be hearing us next time on the Acima Development Podcast. Thank you.