by Haytham ElFadeel - hfadeelm@gmail.com May 2, 2026
You can have a productive week, improve every metric on the dashboard, and still be moving in the wrong direction. The team ships, the model gets better, and the roadmap looks reasonable. Meanwhile, the problem worth solving may have changed.
That's what draws me to the advice of Jim Keller, Andy Grove, Richard Hamming, and others. They approached different problems, but kept coming back to similar questions: What actually matters? Why does the current approach work? What would make us change it?
This post brings together some of the ideas I find most useful from them, alongside my own notes on research and engineering.
1. Understand the mechanism
“They use a transformer.” “They switched to diffusion.” “They added RL.”
These statements tell us something about a system, but much less than we assume. They identify an architecture, a modeling approach, or a way of learning. They don't tell us what the system actually learned, or why it works.
John Jumper makes this point when discussing AlphaFold: we like putting systems into familiar boxes, then treating the highest-level label as the explanation for their success. Calling AlphaFold 3 a diffusion model is technically correct, but that description can obscure the important work happening throughout the system. The label becomes a shortcut past the thing we wanted to understand. (Interview)
His example from AlphaFold 2 makes this concrete. People associated its success with equivariance - the way its geometric computations respect rotations and translations. Yet removing the invariant point attention component produced a relatively small loss compared with the overall improvement over AlphaFold 1. It helped, but the ablation suggested there was more to the story. Other choices, including the representations and loss function, were much less visible in the story people repeated.
A component can be important, recognizable, and technically interesting without explaining most of the result.
Suppose an ML model improves after a team introduces a Transformer. Several things may have changed together: which objects can exchange information, how the input is represented, the training data, and the optimization procedure. Saying “Transformers understand interaction better” skips the interesting part. What information was previously unavailable? How does it propagate now? Which change was necessary for the improvement?
Taxonomy gives us a vocabulary. Mechanism gives us a way to reason.
Jim Keller makes a related distinction between following a recipe and understanding the process behind it. A recipe can produce excellent bread. Understanding the process helps when the ingredients, equipment, or desired result change (interview).
Engineering teams accumulate recipes too: add this loss, tune that parameter, put a special case here. Those recipes are valuable, until the problem changes and we aren't sure which parts still apply.
Before committing to an important design decision, I'd want to answer four questions:
- What is the mechanism that makes this work?
- What evidence supports that explanation?
- What assumptions does it depend on?
- What observation would show that those assumptions no longer hold?
The same applies to business. “This is how successful companies do it” is a starting point. We still need to understand why it worked there and whether those conditions apply here.
2. Choose problems with consequences and a way in
Hamming's advice sounds obvious, until we look at what we actually spent your week working on:
“If you do not work on an important problem, it's unlikely you'll do important work.”
But the problem also needs a reasonable way in: a plan for making progress and some reason to believe it could work. Hamming makes this point in You and Your Research.
There's a trap on the other side too: choosing work only because we already know how to do it. A familiar problem comes with a credible schedule, known tools, and an easy explanation in the next review. An important problem may initially offer none of those conveniences.
A lot of wasted effort comes from not spending enough time with the problem. We notice something going wrong and jump straight to solutions. Having an answer feels productive; admitting we haven't understood the problem yet is less satisfying.
Obi Felten put it well in a 2014 talk:
“we reward people for problem solving, rather than problem stating.”
“We need more engineers,” for example, already assumes a solution. If projects are stuck waiting for decisions, hiring may just give us more people waiting. A few days spent following the work and understanding where it gets stuck can save months of fixing the wrong thing. Before asking how to solve it, it's worth asking whether we've understood what “it” actually is.
Hamming also describes researchers keeping several important problems in mind, ready to recognize when a new idea makes one approachable. I like that idea: keep a short list of problems that matter, even if you can't put them on the roadmap yet.
For each, write down what currently blocks progress. Missing data? An unreliable measurement? Compute cost? An assumption nobody has tested? When something changes, revisit the blocked problem. A new technique becomes much more interesting when you already know what you need it for.
3. Know which S-curve you are on
An approach often begins with slow progress. The team learns how to make it work, improvement accelerates, and eventually the gains become smaller. More effort still helps, but each improvement gets harder.
In The S-Curves of Computing, Jim Keller and Pushkar Ranade describe how sustained progress can emerge from successive technologies, each with its own period of diminishing returns. A smooth growth curve from a distance can hide a series of difficult transitions underneath.
The hard part is that the replacement may initially lose. You are comparing a mature system with years of optimization against a prototype with unfinished tooling and plenty of bugs. Today's score won't tell you which has more room to improve.
The appeal of a new curve is both how fast you can improve and how far you can go. On a mature curve, months of work may buy a small gain. Once a new approach gets going, that same effort may produce much larger improvements, with more room left to grow. A prototype can be behind today and still be catching up quickly.
Think of BlackBerry and the first iPhone. BlackBerry had spent years refining the keyboard-and-email experience. The iPhone gave up the physical keyboard and initially lacked native Exchange support. But its large touchscreen, software interface, overall architecture, and later app platform opened up different possibilities for what a phone could become. In hindsight, the iPhone was beginning a different S-curve.
The question is whether you are comparing two points on the same curve, or two approaches with different rates of improvement and different limits. In the same interview, Keller discusses developing a replacement while continuing to improve the existing system.
But the winning curve is much easier to spot afterward. At the time, the next curve is still a hypothesis. A promising prototype has to work under real constraints.
4. Make it easy to discover that you are wrong
Hamming describes a difficult balance: believe an idea enough to work on it, while remaining alert to the evidence against it. Without commitment, little gets built. Without doubt, a failed hypothesis can become a career.
Before a large investment, I'd write down what evidence would change the decision. “We will follow the data” is easy to say after choosing which data to show.
Suppose the hypothesis is that more scale will solve a particular failure. Define that failure, measure it separately, and check whether it improves as scale increases. If the aggregate score rises while that failure stays flat, the original hypothesis still needs an answer.
Grove points out that people close to customers and technical changes can notice trouble before senior management does. He explores this in Only the Paranoid Survive, especially when the conditions that made a business successful begin to change.
A review should leave room for someone to say, “This result doesn't fit our explanation,” and be taken seriously. People need to be able to challenge the reasoning, and everyone should know who makes the final decision.
Grove's outsider thought experiment is helpful here: imagine a new leader inheriting the situation without your attachment to the previous decisions. What would they change? Then add the practical constraint: what would that change cost from where we actually are today?
That second question matters. Emotional attachment can keep a bad system alive, but migration costs are real. Good judgment has to account for both.
5. Build judgment beyond yourself
Grove's central idea in High Output Management is that a manager's output comes through the output of the organizations they supervise and influence. That changes what counts as a productive day.
Making ten decisions yourself can feel productive. If the same ten decisions return next week, the team may have learned very little.
Consider a technical lead who approves every experiment. At first, this improves quality. As the team grows, everyone ends up waiting for the lead. The next improvement might come from teaching people how to formulate a hypothesis, choose a baseline, and interpret an ambiguous result. The payoff appears in decisions the lead no longer has to make.
How much support someone needs also depends on their experience with that particular responsibility. Grove calls this task-relevant maturity. An excellent researcher taking on their first production launch may need detailed support. Someone who has successfully owned several launches may need clear objectives and occasional checkpoints. Seniority alone doesn't tell you which.
This also connects back to Keller. If we teach people only which decisions to make, we distribute recipes. If we explain the assumptions, tradeoffs, and evidence behind those decisions, we help them adapt when the problem changes.
For me, a useful leadership question is: What will the team be able to do after this interaction that it could not do before?
Sometimes the answer is a resolved blocker. Sometimes it is a better way to think about a whole class of problems. Both matter; the second keeps paying off.
6. Develop research taste
Research taste shows up before you have a result. It is the ability to notice which question deserves attention, which experiment could tell you something important, and which promising direction is mostly an expensive distraction.
One way to practice is to pause after the problem statement when reading a paper. Given the problem and the constraints, what would you try? What would you expect a simple baseline to do? Which result would surprise you? Then read on and compare your thinking with what the authors actually did.
Hamming recommends something similar: get the problem clear and spend time thinking about your own approach before reading other people's solutions. He discusses this in the questions at the end of You and Your Research.
But a finished paper usually makes the path look cleaner than it was. You rarely see all the dead ends and months of confusion. Ask good researchers how they chose the problem, what they tried first, and what made them abandon an approach. Their discarded ideas can teach you as much about their judgment as the final result.
You also need a few experiments of your own. Reproduce a surprising result, test an assumption everyone seems to repeat, or build the simplest experiment that could distinguish two explanations.
A useful test of a research idea is: If this experiment works, what will we learn? And if it fails, will we still learn something worth knowing? Be specific about both. An experiment that leaves you equally confused either way probably needs a better design.
7. Improve judgment by keeping track of your decisions
It's easy to remember that you were right about a project and forget that you were right for a completely different reason. Once we know the outcome, the original uncertainty becomes surprisingly hard to remember.
For a decision that matters, write down what you expect to happen, why you expect it, which alternatives you considered, and where you're least confident. Put a date on when you'll revisit it.
Suppose you approve a project to make training twice as fast because you expect the team to run more experiments. The project succeeds technically, but the number of experiments barely changes. It turns out the team spends most of its time waiting for data and interpreting results.
The performance estimate was right. The assumption about what limited the team's progress was wrong. If the review stops at “we delivered the speedup,” you miss the part that would improve your next decision.
When you look back, separate what you could reasonably have known from what only became clear afterward. A reasonable decision can fail, and a careless decision can work out. Look for patterns across decisions: do you repeatedly underestimate integration work, overestimate how quickly a team can adopt a new tool, or trust results from settings that are too different from your own?
Being good at evaluating model architectures doesn't automatically make you good at estimating launch schedules or recognizing customer demand. In a new area, give your intuition less weight and seek people who have seen the relevant failures.
The next time a similar decision comes around, you'll have more to go on than a vague memory of how the last one turned out.
8. Keep the interfaces simple enough to understand
One detail in NASA's What Made Apollo a Success? stands out. George Low describes roughly 100 wires connecting the Saturn launch vehicle and the Apollo spacecraft. Keeping that interface small meant that one engineer could understand it and reason through the effects of changes on either side.
The point is to make changes easier to reason about. If nobody understands all the ways two parts depend on each other, it's hard to know whether changing one will break the other.
Low points out that this also helped the two organizations work more independently. The same applies to software teams. If two parts of a system depend on each other's internal details, even a small change can require another meeting. A clear agreement about how they connect gives each team more room to work on its own.
Imagine two teams building a robot. One writes the software that spots obstacles; the other writes the software that decides where to move. They need to agree on what “two meters ahead” means: ahead of the camera or ahead of the robot? They also need to know how old the measurement is and how reliable it is. Each team's code can pass its own tests, yet the robot can still make the wrong move because the teams understood the same message differently.
At that boundary, I'd want to know:
- What exactly does one side promise the other?
- Which assumptions have we left implicit?
- Who checks the consequences when either side changes?
Keeping an interface simple does not mean removing information that the task requires. It means making the necessary dependencies explicit and manageable. A short message with five undocumented assumptions can be harder to understand than a longer, well-defined one.
Before adding another coordination meeting, it is worth checking whether the design itself is creating the need for that coordination.
9. A Friday afternoon check
Hamming reserved time on Friday afternoons to think about where his field was going. He sometimes found that what he believed mattered and what he actually spent his week doing had drifted apart. He describes that practice in You and Your Research.
I like this as a small habit to take from all of this. A few questions for the end of the week:
- Which important problem did we move closer to solving?
- What did we learn about why our approach works or fails?
- What evidence made us less confident in our current plan?
- What should we now start, stop, or change?
The answers don't need to be dramatic. Sometimes the right decision is to keep going. Sometimes a small anomaly deserves an experiment. Sometimes the team needs clearer ownership more than it needs a new architecture.
But if our view of the future changes while our calendar and resource allocation stay exactly the same, we should be able to explain why.