Your Limiting Factor (in Software)

... is you ...

I’ve never been one who’s much for self-help, you know, all those books you walk past in the airport or at Barnes and Noble. “10 Ways to Transform Yourself …” Yeah, right, I usually snort to myself and walk on.

Maybe I was wrong all along, or maybe it was because I was raised in a cornfield, and nothing was handed to me besides a fist full of dirt. Either way, the only kind of self-help I believe in is “Put The Work In, " which is the title of my self-help book.

Today’s article is less about software and more about the art of becoming a …

• better software engineer

• more senior engineer

• smarter engineer

• expert in <insert technology>

• someone to be reckoned with

Maybe this is my version of self-help for Software Engineers 101. What do you think? Would you like to buy a copy for 9.99 on late-night TV?

I was inspired to write this while lying flat on my back in my unfinished basement, pumping a set of dumbbells up and down for the millionth time.

I hate to break the bad news to you.

I'm in a somewhat undesirable position in that I’ve been writing to all of you for years. I get many inquiries from various people about everything related to data engineering and programming. I’ve also had the unfortunate or fortunate position of being able to work at various startups over a wide distribution of teams, giving me a first-hand view of the foibles of the software masses.

There are always outliers in every dataset, but generally, I find the following to be true.

You are your worst enemy, and your limiting factor is yourself.

At the risk of simplifying too much, in the age of AI and ChatGPT that can write all your code for you … you are at risk of simply not putting in the work.

Let me ask you a question. Think of the one technology you work on a lot … such as Spark, Databricks, Snowflake, Kubernetes, AWS, GCP, Postgres, Rust, or Python….

Next, think about someone you “know” or are aware of who seems to be “an expert” or “really good at that thing.”

Do you really think that knowledge simply fell from the sky into their ears and slowly made its way to their brains, making them experts overnight?

You, my friend, breathe the same air as everyone else. Sure, some people are “smarter” than others, but generally speaking, when we look across the mass of Software Engineers pumping out code on a daily basis … there is a relatively small set of things that set the “good ones” or “experts” apart from the mediocre.

I fairly regularly get contacted by someone(s) about getting into creating content, especially this Substack, folks looking for advice, tips, secrets, and encouragement to do it themselves. I always let them in on some incredible insights that are mind-blowing (insert sarcasm). I think they apply to software as well. Here, I’m giving them to you for free.

• Consistency over long periods of time is #1

• Find what you’re good at and double down

• Learn the things you don’t want to (marketing, etc, aka soft skills)

• Doing something is better than nothing or perfection.

If you want to be a good writer … then write. If you want to be better at Rust … then write more Rust. If you want to become a master at Spark, then write more Spark and read and study more about Spark.

Do hard things and grow yourself.

Life is like that inescapable bell curve from college or high school, good old data distribution. Why do you think everyone hangs in the middle, just doing their thing?

Because it’s easy, that’s why. You can just show up every day and do your job, your little pipelines, your Python and SQL.

But, if you want to move over to the right side of the bell curve, well that takes a little more effort. Surprise, surprise.

How many things have you started and stopped in life? Wanted to start a business, build that cool project, write that book, start a YouTube channel, get in shape … hmmm?

Not everyone does those things or moves over to the right because that is a hard path. A path filled with extra work, less free time, filled with blood, sweat, and tears.

I’m not saying you can or should be good at everything, that isn’t possible. It is possible to be good at some things, nay, better than most at some things.

• Learn some new things, be comfortable being uncomfortable

• Be ok with being you

• Put in extra work and time into your craft OUTSIDE the 9-5

• Set realistic goals for the year

Don’t have a classic victim mentality. There is always a reason why you can’t do something. Or you could just do that hard thing anyway and reap the benefits. It’s only rarely that hard work doesn’t pay off.

Special note for Software Engineers

I want to end this little pep talk with a special note for all you Software Engineers pumping out tech debt every day. By all means, get better at writing code. You must.

Learn new languages, tools, you name it, shoot for the technical stars.

But … never forget to work on your soft skills. This is the single largest limiting factor I see on a regular basis.

I could point you to a myriad of brilliant Software Engineers who languish in obscurity, always unsure why noone recognizes their talent for what it is.

• It doesn’t matter how nice your code is if you are an ass to work with.

• Business value matters, so do communication skills.

• Be able to see beyond the code to the bigger picture.

We all provide software and solutions in the larger context of a business that needs to operate at profit and make tradeoffs based on supporting internal and external customers.

The code you write isn’t the most important thing in the world.