JSON with Rust

Thoughts on being Explicit vs Vague

While recently working on a side project to make myself rich off the stock market using Rust, I ran into a problem of sorts. Well, not a problem per se, but a conundrum.

I had a choice to make, not unlike choices we have to make as Data Engineers writing code all day long. To be explicit or vague. I mean if you are in a hurry just take the easy route?

It seemed wrong.

In this particular instance, I was using Rust to process some JSON. I think this will make a great example for us to talk about Explicitness vs Vaugeness, regardless if you’ve even written Rust or not.

Moving fast … or correctly. Vague vs Explicit.

As someone who’s been forced to use Python to write the majority of my production code over the years, say it ain’t so, decades … there are certain downsides. Namely, being vague with code because you have no other choice really.

This isn’t something I endorse and still don’t like. I enjoy code to be explicit. Say what you mean. Be verbose.

Instead of starting with Rust let’s start with our odd cousin still living in Aunts’ basement … Python.

I was recently working with a JSON file that is a list of tickers for publicly traded stocks. It looks like this.

How would most people read this JSON with Python? Probably something like as follows …

This probably looks very normal to you. But, something is missing if we were working in a real codebase where some of this logic is encapsulated …

• read and file I/O in one spot

• processing the JSON in another

A few things I wouldn’t know looking at the code if I was making changes, additions, or whatever to this code.

• I don’t know what the actual structure of JSON is or should be.

• I don’t know data types.

Sure, in Python I could add some extra libraries like …

pip install jsonschema

and then go about the business of writing some validation code etc. Most people never do this, it’s Python.

Reading JSON with Rust.

I mean I did say this was a Rust thing in the beginning, and it is, I just wanted to set the stage. Now that we have set the stage with our dirty little Python code, I’m sure you’ve heard people, including me, laud the safety of Rust and its wonderful Type system.

This is true, the static Types at compile time with Rust is one of the reasons many people love it, and it’s probably why it’s quickly becoming the de facto standard for backend Data Engineering systems.

Anyway, I digress. Back to the problem at hand. Rust + JSON.

Let’s say I wanted to read this same JSON file with Rust. How might I go about it? One would expect, considering Rust’s ideals that this would be verbose code.

Let’s try.

First, let’s set a struct and type to hold our and prove upfront what our JSON will be once we load it.

In my case, I was loading the JSON and then pushing the results of the above JSON structure into a SQLite database.

Basically, what is happening …

• Read file

• Use serde_json to get results into the TickerMap (which includes our Ticker struct of course)

• Iterate those tickers and access the struct attributes with dot notation.

I mean it does feel very natural to Rust to do it this way. It’s very Explicit and not Vague at all, just what would we expect.

Unlike Python, we aren’t sort of hoping that things work out unless we add the extra package and force-check the schema. It’s sorta built-in with Rust, isn’t it?

We are kind of pushing possible problems back as far as we can in the code, getting things to pop earlier if they are not what we expect. Also, the code is easy to reason about out of the box.

Writing lazy Rust.

But wait, there’s more. Can we be lazy in Rust and be more like Python? Just load up who knows what JSON and hope for the best?

Sure we can. If we use … use serde_json::Value; we can write the below function.

Going this route we can delete the Struct and Type we used before. I mean it’s code reduction right??? Less code, less problems???

I guess when we look at this sneaky Rust code it does seem a little more obtuse to me, compared to the verbose first example. At least for me.

I just don’t like how the second option feels. It feels like cheating, it feels like I might as well use Python if I’m going to write Rust like that.

The Programmer vs the Program.

I guess after all this hullabaloo it brings me to the point of the whole thing.

Sometimes I forget it’s just as much about how the Programmer decides to solve the problem as much as it is about the semantics of a particular language, in our case Python or Rust.

We all like to worship at the altar of something, after years of Python I like to spend my time Rusting my way through data. It’s a joy to write and when it compiles I get a warm feeling in my tummy much like that first swig of a Pale IPA.

Rust does a better job than pretty much any other language to date in protecting us from us. That’s why it’s popular. But at the end of the day, the developer is the developer and they are going to do what they are going to do.

I still find myself in the camp of being verbose. Use Types, be explicit when you can, and be verbose. I’m probably just getting old and grumpy but me thinks not enough emphasis is put on readability and debug ability in codebases.

In the age of ChatGTP and Copilot, the draw to be lazy is ever near and calling your name.