How to deal with ambiguity.
If you have no idea what to do, what do you do?
Nothing strikes fear in the heart of some freshly minted developer than sitting in a room full of apparently intelligent people, who jibber and chatter like chimps in the zoo at feeding time. Everyone is talking fast, throwing terms, knowing nods, and mumbles and grumbles with insider looks and agreements. Everything seems important, and obvious to the crowd.
Finally, everyone turns their burning and perceiving gazes in your direction, asking you, “So, what's your estimate, how long is this going to take?”
Fumbling for words like you just swallowed a lump of sawdust, you wave your hands back in forth and mumble something about a week or two. Exhausted, you slump back in your chair, sweating and feeling like you just ran a marathon. You want to crawl into the nearest hole and already wonder what you’re going to share at tomorrow’s standup.
Ambiguity. The destroyer of confidence, projects, and software engineers.
Thanks to Delta for sponsoring this newsletter! I personally use Delta Lake on a daily basis, and I believe this technology represents the future of Data Engineering. Check out their website below.
When you don't know what you don’t know.
When you are new to your career, or job, it’s a pretty normal feeling, being lost that is. Pretty much nothing makes sense, at all. It’s strange, you go from being a confident human being to feeling like a helpless baby. You probably don’t like it, your body doesn’t like it. It’s not fun for anyone, although some people probably deal with it better than others.
Maybe you’ve been around awhile, but something new gets dropped in your lap from Product. There’s a good chance that you can tell that Product Leader doesn’t even know what they want, shapeshifting between one meeting and another like some dark elven magic. Doesn’t matter. It’s all the same problem, ambiguity.
This situation seems to be especially true for software engineers and developers, as per the nature of our chosen profession, tools, tech, and approaches change as quickly, and are never stable or last.
It can be quite difficult to tackle a project when you don’t know …
The business context.
The toolset or architecture.
Who anyone is (relationships).
Where to look for answers.
What the expectations are.
Ambiguity, ambiguity, ambiguity. That’s just life in software, but how can we overcome such things, is it possible to thrive in an ambiguous environment? I think so. I believe there are a few simple steps anyone can take to ferret out enough information to become a valuable member, the unicorn who doesn’t need a lot of hand-holding and is able to produce results and iterate as needed.
It starts with a question. “Ask and ye shall receive.”
Questions, it all starts with questions. Instead of stewing in the unknown, and thinking about all things you don’t know, start by asking a few questions. Part of what separates a Senior Engineer from one of them there “Engineers” is the ability to ask questions, and the right questions.
It seems kinda obvious, but when you have ambiguity, some of that is by design, the product may be in flux and there might be no solid answers, but some of the ambiguity is most likely because something is locked inside someone’s head, tribal knowledge, and needs to be let loose. Let loose via questions.
I suppose it’s hard to generalize, but when it comes to ambiguity in Data Engineering projects, maybe we can come up with a list of questions to ask. Starting with the basics and diving deep. To make sense of what you don’t know, and find out about what you don’t know, ask questions.
Where does this data come from, and what is its source?
How often does this data come, and what is the frequency?
What person knows most about this data, can I talk to them?
What is the exact desired state?
Which business person (Product?) is in charge of this product, and has a vested interest in this project?
What is the timeline for this project?
Who knows the most about this or that tool?
I guess the list will never end, but you have to start somewhere. Start with a lot of questions, the answers or the “I don’t knows” will give you hints and snippets of what’s going on, this can help you map out the big picture. Once you start sketching out the big picture, you will feel like slowly your world is coming into shape and form.
Start with what you do know.
You may not know everything, but you probably know something more than you think. Surprise, Surprise, the older and wiser you become, hopefully, the more you realize that no one knows everything. Part of being successful in Software Engineering is realizing it’s a team sport, the best products built are because a lot of good people work together to achieve a desired end.
Sometimes taking small steps can give you a feeling of confidence, help you feel like progress is being made, and actually is progress. Not moving forward for fear of not knowing can be a disease that slowly eats and rots the apple from the inside. From a Data Engineering project perspective, what can you do to start moving forward?
If you have a lot of ambiquity in your project, start working on what you know, as you move along some things will become more clear, some less, you will end up with a clearer picture of where the gaps of knowledge are.
Others will see your hardwork, clarity, and desire to get things done, inspiring them to step in to help, and answer questions.
Closing Thoughts.
I’m a big fan of action over inaction. If you don’t know something I think the best course of action is to attack, and move in a direction. Getting caught up in the not-knowing and the endless cycle of not getting anything done, relying on others to solve your problems … well that is a problem.
Of course, we have to rely on others for help. But we can still seize the day.
It’s typically better to do something than nothing, and honestly, all we have to do is apply a little rigor and logic to moving toward and we can slay that dragon of ambiquity.