I’m not talking just about people simply handing over their email inbox to OpenClaw. I’m referring to major incidents like how “AWS suffered ' at least two outages’ caused by AI tools.” 

Code is cheap now, and we can fully delegate it to AI, but coding is only a small part of our jobs. The others, like handling incidents caused by AI code, are not.

In all the situations below, you'll notice a pattern: people think “AI can handle most of it, so why not all of it?” and here’s how that leads to disaster.

Hiring

The misuse of automation in hiring predates the rise in LLMs.

Eleven years ago, I applied for a Django role and got rejected within two minutes at 01 AM, because I needed to know more about “Python” for the job. The email seemed to be written by a person. I submitted a new application with just one word added and received an interview invitation… The rejection was because the scanner didn’t find the word ‘Python’. 

The main problem with companies that pull “clever” stunts like these is that they exclude great candidates. Not only that, but people will notice your flaws and share them publicly on platforms like Glassdoor, which can tank your reputation.

Some argue that automation is necessary because applicant volume can become overwhelming. I disagree. During the COVID hiring surge, I reviewed over 1,000 resumes a year and never considered automating screening.

The reason why you shouldn't automate hiring is that it is the most important thing you do.

Hiring well is the most important thing in the universe. […] Nothing else comes close. So when you’re working on hiring […] everything else you could be doing is stupid and should be ignored!

— Valve New Employee Handbook

Even with 300 applicants each month, you can review all the resumes in less than an hour by using better judgment than AI. That one hour spent is more valuable than dismissing a potentially great candidate. Finding the right candidate early also reduces the hours spent on interviews.

Now that people are embedding LLMs into the hiring process, the situation has worsened. I see many pitches for tools that claim to be better at evaluating candidates’ interview performance than a human, which is simply absurd.

Hiring is a human process: you need to understand not only what they say that makes sense, but also what excites and motivates them to see if they’ll be a good fit for the role. 

You can’t measure qualities like enthusiasm and soft skills with AI. It will only accept what the candidate says at face value.

A candidate might claim they are passionate about working with bank accounting software in Assembly at your Assembly bank firm, but are they really?

Code reviewing

From my personal experience with AI review tools like CodeRabbit, Claude, and Gemini, I've noticed that a pull request with 12 issues results in 12 comments, but only about 6 are actual problems. The rest tend to be just noise or go unaddressed.

This doesn't mean those tools are useless. Letting them do an initial pass is very helpful, and some humans wouldn't catch some of the issues they find, especially the deep logical problems.

The issue with automated review tools is that they are becoming the de facto gatekeepers for deploying code to production, leading to future outages and a low-quality codebase. The inmates have taken over the asylum, and we now have AI reviewing code generated by AI.

Review tools are very focused on checking whether your PR makes logical sense, such as whether you forgot to add auth behind a route, but they can't, for example, judge whether your code worsens the codebase.

They can't raise the bar, which is the best part of human reviews. Every time we create or review a PR, it's a chance to learn how to become a better engineer and to leave the codebase in a better state than we found it.

Comments from peers like “you are duplicating logic, you should DRY these components” encourage us to review our own code and improve as engineers. Relying only on AI review takes away that chance.

Most incidents I observe happen because AI struggles to evaluate second-order effects; it overlooks the Chesterton fence. For example, if you try to delete or change a downstream parameter, like a parameter needed and was removed by an LLM, which wasn’t caught by linting. This reflects a limitation of current models: they can't review your code across repos.

Technical Writing

I'm tired of reading AI-generated writing: it just doesn't respect the reader's time. I see many AI-produced texts that could be shortened by a quarter without losing any important information.

Reading emails, meeting notes, or technical documents filled with emoji spam and strange analogies (“it's not X, it's Y”) is tiring. When I see the words “Executive Summary,” I often hesitate to read it.

I would have written a shorter letter, but I did not have the time.

— Blaise Pascal

There is power in simplicity and in respecting your reader's time. Most of my blog posts are cut by 50% just before I publish them.

Most people I know who use AI for communication do so because they believe their writing is not good.

But honestly, the goal of communication isn't grammar skills but to get the point across. 

Good grammar is often overrated anyway. One of my favorite documents is the leaked MrBeast memo PDF, which is full of grammatical and punctuation errors but clearly communicates its message through a “braindump”, much better than any LLM ever could.

Consumer research

When you ask an LLM about your roadmap, you're likely querying what countless other companies with very different issues have already tried. The AI relies on patterns from its training data, and in my experience, those patterns tend to be too generic compared to the insights of a seasoned domain expert.

If your software is meant for hospital accountants, do you think they take time to blog about the frustrations of their workflow? The knowledge is stored in their minds, and you need to extract it. This vital knowledge is never documented and thus never accessible to an LLM.

I spent three years researching and working on accessibility for nonverbal individuals. If I ask the AI about what this industry lacks, it will start discussing the need for better UX solutions (there are countless papers on this, I even naively wrote one). Still, I saw multiple companies enter the market with great UX products only to crash and burn.

After a while, I realized that poor UX apps still dominate adoption because these companies invest millions in lobbying, partnerships with insurance companies, and training, which is the thing no one talks about.

Management tools

I get many messages from bots on Reddit and LinkedIn about AI management tools, but as I mentioned before, they lack context. The worst part is that they think they can make judgments with the limited context they have.

Here’s an example of a feedback tool output:

“This engineer sucks, they do 40% fewer PRs than the median, I marked him as an underperformer… I also told your boss, HR and CTO about it, better do something!”

- Some tool with a fancy name and a “.io” domain

But yet, that engineer is one of the best I have worked with.

The issue is that they try to outsmart the manager, which leads lazy managers to use the AI's suggestions as an excuse, resulting in poorly thought-out feedback because “The computer says no.”

So, what should you delegate to AI?

Think of the current LLMs as an “added value tool”, not a product, and definitely not an expert.

Most of what I wrote above is problematic because it overestimates what LLMs can do and enables them to operate unsupervised. You can't go back in time after AI makes a mistake, and there are no guardrails once a mistake is made.

I received a lot of criticism for my post about using AI to select E2E tests in a PR pipeline. Yes, it sounds crazy, but this is the “added value” part: if the AI fails at selecting the right test, we will catch it before deployment. The value provided is that having it is better than having no pre-checks at all.

Before giving AI control, ask how resilient our system is when (not if) the AI screws up, and ensure you have stronger safety nets before delegating completely.

But the question is, how does that impact everything else?

In most teams, coding - reading, writing, and debugging code - used to be the part that took engineers the most time, but that is no longer the bottleneck.

The Theory of Constraints states that every system has a bottleneck, since without one, it would operate infinitely fast, which is impossible.

Let's see the new bottlenecks, their effect on the size and roles of engineering teams, and why Amazon's two-pizza-team rule - “teams should be small enough to be fed by two large pizzas, ideally comprising 5-8 people” - is being phased out. 

Product and design are the new bottlenecks

Currently, LLMs are less useful for product managers and designers than for engineers.

On the Designer side, they struggle to create great prototypes. You notice how every AI product homepage looks the same. I believe this is a limitation of LLMs: they tend to generate ideas near the middle of their training data’s bell curve, which prevents bad design but also limits truly innovative concepts.

On the Product Manager (PM) side, LLMs can gather data and insights, but the most time-consuming part for the PM is communicating and talking with clients, which can't be automated as effectively.

This creates a new bottleneck: The project output starts to depend on the delivery speed of product specs and wireframes.

And it gets worse: generally, you have 4-7 engineers but a single (sometimes shared) PM and Designer in a team, creating an imbalance.

Sharing the Legos

Some companies recognized this imbalance and asked, “What if the engineers were involved in Product and Design instead of just receiving the product specs?” Then they started hiring Product Engineers. 

This is not a new concept; the role has existed for over 16 years. But I am highlighting this because they are now more relevant than ever.

Okay, so what are product engineers? They are software engineers empowered to handle some responsibilities of PMs and Designers, balancing the roles.

Product engineers assume traditional PM roles, including owning the roadmap, engaging with users, analyzing data, framing opportunities, and determining what to build. However, they do not replace a PM. The PM still provides context but is no longer the main driver of implementation.

On the designer side, they also assemble the building blocks of a design system. The designer still creates those blocks and collaborates on the UX flow, but they are no longer responsible for producing “pixel-perfect prototypes” that engineers must follow.

The rise of specialists

Not everyone should or will be a product engineer.

But the traditional software engineer who is "A jack of all trades, master of none" will not surpass an engineer who is a “master of one”.

AI is good at producing code of acceptable quality, but it is rarely excellent. You can't simply prompt and merge it into a mature codebase without in-depth human review.

There are many problems with AI coding:

• AI often approaches coding carelessly, neglecting second-order effects: it modifies or removes essential code without considering the consequences.

• For an LLM to fix bugs, they need to make sense (and for the ones that pierce through the abstraction layers, they rarely do), so you can't just have a team entirely using AI without a solid understanding of the tools they use.

• AI will replicate destructive patterns in your codebase, causing a decline in code quality over time because no one will detect and fix them.

So, we need specialists to manage the platform code. It’s not that they won’t use AI, but they will act as gatekeepers during reviews and prevent bad patterns from being merged into the codebase.

We will probably see fewer full-stack engineer openings and more roles for back-end and front-end engineers. This doesn't mean they will do only one or the other, but they will be expected to be an expert in one area.

Small Teams, Big Dreams

Besides the previously mentioned imbalance, large teams also face two main challenges: communication overhead and the need to divide work so team members can work independently toward a shared goal.

For the second problem, it wasn't too bad. We could divide the work into epics that would take an engineer two weeks to finish, and they could work on them independently. But now two weeks feels like a long time.

Another issue is that AI performs best when given a wider context for a problem. This, in turn, makes dividing work even more difficult.

The ideal team size now appears to be 2-3 engineers per project. Even with a larger team, you can divide it into smaller groups of 2 engineers for a set period and observe how quickly they progress. 

However, avoid assigning large projects to a single person. Working alone for an extended period without anyone to brainstorm, review code, or collaborate can be challenging.

From my experience, an individual contributor's frustration tends to increase over time when they are on their own. Also, developers need to maintain and improve their communication skills, which are essential for growth.

The engineering manager role is still needed, but has changed

I get many DMs on Reddit and LinkedIn, pitching AI manager tools that track “productivity” and use AI to “evaluate” an engineer’s performance.

Most fail at the concept stage. Manager performance tools aren’t new, but they all fail by trying to outsmart the manager. AI will never have even 40% of the context a manager needs. It can only evaluate “quantitative” metrics.

But managers who don’t code will be rare: With smaller teams, their responsibility with the People pillar decreases, freeing up time for the “Programming” pillar.

I already explained that the “well-defined engineering manager role” is a myth, and engineering management always involves adapting to a team, so that will always remain the same.

AI makes it easier for engineering managers to participate in coding. They already know how to break down a larger problem into smaller, reviewable goals, just like any senior engineer would.

So you prompt Claude, attend a meeting, and then review the code when you return!

But the role of a manager will differ from that of an engineer. They will not be assigned the same tasks because situations that demand a manager’s attention will always take priority. So it’s crucial to avoid work that could potentially block the team if halted.

We won't stop there

I believe this is just the first wave of changes.

My post assumes that AI won't improve much beyond its current state, which seems like a safe prediction. The progress isn't primarily from better AI models but from how we use them (tools, thinking capacity).

We're also seeing significant investment in designer and PM tooling where AI is not a centerpiece but a complement. I wonder how much more they can still improve.

There are many unknowns I haven’t addressed because they’re still uncertain, such as QA: How much can AI take over QA, and what will the roles of the QA Engineers be?

What else do you think is going to change? If you know, drop a comment.

I didn’t realize at the time, but he was dooming me to be an engineering manager.

Since then, I’ve worked across four companies as a manager, one as a founder, and another as a manager of managers. I will skip the standard advice and lessons on Engineering Management and focus on the non-obvious ones.

1. The “well-defined engineering manager role” is a myth

There is no standard definition for an Engineering Manager. If you pick two random managers, they can do very different things, even if they are at the same company.

In every company I’ve worked at, my role has never been the same. The only constant is that it’s defined by the team's needs, requiring you to balance across four pillars: Product, Process, People, and Programming.

Some examples:

• Large team? Say goodbye to programming. You’ll focus on building careers, coordinating efforts, and navigating the organization to get resources for your team.

• Small team? You’ll manage scope to match reality, and with less communication overhead, you might actually do some coding.

• No PM? You own the product completely: validating features, prioritizing the roadmap, and talking to clients. This takes up most of your time because shipping features that don’t offer user value makes everything else pointless.

• Reporting to the CEO? You’re now the link to sales, operations, and client communications.

The key is to identify where your team’s bottleneck lies in your software development lifecycle. You’ll probably shift between pillars as circumstances change, and that’s the point: the role requires flexibility.

Tip: Never ask the interviewer what they expect from a manager. Some managers assume their experience is industry standard and might find that question odd. Instead, ask about their daily life and the challenges that take up most of their time.

2. Everyone needs to care about the Product

A few times in my career as a developer, I wondered, “Who is this feature even for? Who will use it?” No one on my team knew. We were doing it because we were told to. Morale was low. We felt we were working on things that didn’t matter - and we were. Eventually, our team disbanded, and engineers scattered across other projects.

The most common reason companies fail is creating products that don’t deliver value to users, causing them not to pay.

“Oh, but I have a PM for that,” you might say. But having a PM is not enough. Everyone needs to care about the product. Your team isn’t paid to just deliver code but to use it to solve problems.

Code is valuable only when it benefits the end user. Sometimes, a no-code integration can outperform a custom solution. At times, it's better not to create a new feature at all to avoid maintaining a system. Teams that understand the problem, not just the spec, can pivot when necessary.

3. There is no such thing as a free lunch when it comes to processes

Every process trades time and attention for reliability or quality. The problem occurs when teams stop questioning if the trade is still worth it. Ceremonies become rituals. Metrics turn into goals. No one remembers why we spend an hour of our lives on this meeting.

Process bloat creeps in slowly. An engineer ships a broken UI to production. Designers complain, managers panic, and suddenly every PR requires designer approval. The whole team bears the cost of a single isolated incident.

Good process serves you so you can serve customers. But if you’re not watchful, the process can become the thing. You stop looking at outcomes and just make sure you’re doing the process right. The process is not the thing. It's always worth asking, do we own the process or does the process own us?

— Jeff Bezos, 2016 Letter to Shareholders

The right process varies based on context: team size, experience levels, and deadline pressure. What works for a mature team might not work for a new one. Keep questioning and iterating. If a process isn’t improving delivery, cut it.

4. Communicating downward requires transparency

Your direct reports are the people who interact with you the most. They look to you for leadership and clarity, and trust that you’ll tell them what they need to know.

That’s why lying or withholding information that affects them causes irreversible damage. They might not leave immediately, but they will resent you.

I have a friend who still resents a manager for a lie told three years ago. They found another company, but they’re still angry about it.

“Trust arrives on foot and leaves by horseback.”

- Old Dutch saying

I've seen some managers describe the role as “a shield that blocks everything from above,” and I disagree. A good manager is more like a transparent umbrella. They protect the team from unnecessary stress and pressure, but don’t hide reality from them.

Telling the team: “Our users aren’t thrilled so far. We need to find ways to better serve them. The project risks cancellation if we don’t.” That’s fair game. They deserve to know.

When you do deliver hard news, state it clearly and focus on how the team will do about it. If you act scared, they’ll be scared too. Your goal is to get them thinking about the next steps.

5. Communicating up requires a strategy

I see managers walk into executive meetings saying, “We’re not sure what to do - maybe A, maybe B?” and then leave with orders to do Z, which doesn’t benefit the team or the project.

Executives can’t think of every possibility in detail - that responsibility lies with you and the person who owns the product (which, as we saw, could be you too). When a problem reaches the executives, it’s because a decision is needed, and they will make one.

People above you have limited time to focus on your specific issues. You can’t info dump on them. If they take a misguided action based on what you tell them, it will be your fault.

If you believe in something, clearly state your case, including the advantages and disadvantages. Don’t expect higher-ups to think for you. It’s okay to bounce rough ideas off your direct manager, but beyond that, refine your thoughts - no one will think harder about your problems than you and your team.

If you need a guideline, a document should be: context → problem → plan / alternatives → what support you need.

6. You are 10% player, 30% coach, and 60% cheerleader

Player (10%): Yes, only 10%. You might take on work your team isn’t excited about, but that matters: CI/CD improvements, flaky tests, process tooling. However, you need to stay off the critical path. As soon as you start handling essential tickets, you’ll block your team when managerial work pulls you away. 

Coach (30%): Your performance as a manager is the sum of your team’s output. Coaching involves preventing problematic behaviors from becoming normalized, such as toxicity, repeated mistakes, and consistent underperformance.

It also means supporting engineers' growth by challenging them appropriately, providing the right feedback, and helping them develop skills they’ll carry forward.

Cheerleader (60%): Praise people more than you think you should. Validation is important. Most engineers prefer feeling appreciated over having a ping-pong table.

But give praise genuinely, not automatically. I once joined a team where retrospectives included 30 minutes of mutual praise - n-squared compliments every week. It felt hollow. Not every week has something extraordinary, and when praise becomes expected, it loses its impact. The hedonic treadmill is real.

Make your engineers’ wins visible beyond your team. Encourage them to pursue impact outside the team, and celebrate their achievements when they do.

Every team operates like a small company within the larger organization. I find that its morale also exists independently of the company's overall morale.

7. Your goal is for your team to thrive without you

Most managers don’t plan to become bottlenecks. It happens gradually. A critical tool needs an owner, and you think, “I’ll handle this for now.” Someone needs to be the point of contact for another team, and it’s easiest if it’s you. Technical decisions keep landing on your desk because you’re the one with context. Before you know it, work stops without you.

If you can’t take a month off and return to a well-functioning team, you need to work toward making that possible. 

You’re too busy to be the bottleneck. If people keep reaching out to you for recurring tasks, delegate by teaching someone else to handle them. Point team members directly to each other or, even better, create group chats to facilitate natural discussions.

Don’t become the bus factor of 1. Train others so work continues smoothly even when you’re overwhelmed or unavailable.

Avoid making people feel they need your permission for small, reversible decisions. Empower them with agency. Request to stay informed about their decisions, but let them handle the technical side.

The reason you will do this is that managerial work can, and will, appear at the worst time. If you are the bus factor, you will be screwing your team when it happens. There are many engineers, but only one manager. Stay accessible for tasks that only you can handle.

8. You can’t succeed without trusting your team

Micromanagers micromanage because they don’t trust.

Ask yourself: can you trust every engineer on your team to do their best without you constantly watching? If not, something needs to change—either in you or in them.

Trust isn’t about just technical skill. If I asked my current engineers (mobile and web developers) to build a Game Boy emulator from scratch, they wouldn’t know where to begin. They’d probably take months (some just weeks). But I’m sure they’d try their best to figure out how to run Pokémon Gold.

You need to trust both their abilities and their honesty:

• If you can’t trust their skills at their level of experience, it’s your job to help them get better.

• If you can’t trust their honesty, and you have good reasons not to, then you need to part ways.

9. Trust, but verify

Even great engineers get stuck without realizing it. Watching progress helps you spot when they need support before others see them as underperforming.

Processes like sprints and OKRs mainly focus on the “verify” stage (see, your manager does this too). They serve as a shared interface to ensure work gets done. This isn’t about lack of trust but accountability.

Verification involves using metrics and evidence. There are two types: quantitative and qualitative.

Quantitative is simple: PRs merged, points completed, code reviewed. You can glance at these, but decisions shouldn’t be based on them alone. If you could gauge engineer performance from numbers, managers wouldn’t be needed.

Knowing the Qualitative metrics shows a manager's worth. “This engineer has fewer PRs, but they’re always watching Slack and hopping into calls to help others.” “This engineer always discusses tickets with product first - their output ends up far better than our original specs.” “This engineer explains complex concepts in ways everyone can understand and makes other teams use our tool better.”

These insights depend on truly knowing your team. That’s why most “management AI tools” are doomed to fail. They only focus on quantitative metrics. They don’t attend your standups, don’t conduct 1:1s for you, and don’t know who’s quietly holding the team together. A good manager does.

10. Eventually delegate everything.

Stop having pet projects; that’s a Staff Engineer’s domain. For a manager, every project is cattle: it needs to be completed, automated, delegated, or cancelled.

Managers hold on to projects for many reasons. Sometimes it’s comfort - you know this system, you built it, and it feels good to stay close to it. Sometimes it’s identity - you want to stay “technical” and not lose your edge. Sometimes it’s fear - you don’t trust it’ll be done right without you. None of these is a good reason to hold on.

The “I can do it faster myself” mindset might be accurate, but in the long run, it’s not sustainable. Every time you do it yourself, you rob someone else from learning and ensure you’ll be doing it forever.

11. There is no free lunch when it comes to reducing risk

Be risk-averse, not risk-paranoid. You can’t account for every variable. Some things can’t be anticipated, and overcorrecting may be worse than the original issue.

Hiring is where I see this most often. After a bad hire, managers start requiring referrals, but almost anyone, no matter how unskilled or dishonest, can find someone to vouch for them. Others add more interviewers to the panel, thinking more eyes means better vetting.

The opposite happens: each interviewer becomes more lax, expecting someone else to be “the bad guy.” Responsibility gets diluted. Three great interviews beat seven mediocre ones.

Think about second-order effects too: while you’re scheduling that seventh round, good candidates are accepting offers elsewhere. The best talent moves quickly. A slow, risk-averse process filters out exactly the people you wanted to hire.

If any of this resonated, my free online work-in-progress book goes deeper. If you’re a manager too, I’d love to hear what you’ve learned. Drop it in the comments!

So when ChatGPT launched, I thought it would finally replace Grammarly.

But LLMs have their own problems:

“It’s not just x—it’s y,”

Rhetorical questions? Affirmative answers!

“Here’s the kicker”: That preface was entirely unnecessary,

And in the end, it ends with recaps — that repeat everything already said, now with bullet points.

The problem with AI text is that when you read it, your first thought is: “Did this person actually invest time in this, or did they write a two-line prompt and expect me to read something they never even thought about?”

And as some people put it: “I’d rather just read the prompt.”

LLMs can’t be genuine because they don’t know how to be a person. They read text from multiple public sources and average it out. They weren’t trained by eavesdropping on authentic conversations or messages. (At least I hope not)

The more the AI creates for you, the worse the output becomes.

That’s why when you ask it to keep it casual, it turns into “How do you do, fellow kids?” and when you ask for a professional tone, it becomes “Alas, who’d’ve done this?”.

What You Need to Provide

If you want LLMs to cook, you need to provide ingredients.

As a general writing (and cooking) tip, start it raw. Don’t use autocorrect. In fact, don’t even look at what you’re typing. Close your eyes and let raw ideas flow, along with grammatical mistakes and misconstrued sentences. Just make it coherent enough. Make bullet points to answer: “What’s the point of me writing this?”

Connect those bullet points with your personality, which dictates how you link sentences. A serious person uses serious connectors; a casual person throws in verbal expressions (and memes).

How LLMs can help

When you have the first draft, the key is using the right edits. The biggest mistake people make is while prompting. If you prompt like a casual writer, it treats you like one.

Saying, "Improve the text below for my email,” makes the AI slopify everything: it accesses the neural latent space of “This person needs my help immensely.”.

You need to signal, “Hey, I know what I’m writing. I just need help improving the flow while keeping my own words.”

You can do this by using the verbiage editors and publishers use during the different editing phases, from solidifying the overall scope to minor edits like correcting grammar.

While the LLM won't write for you, it can help you immensely, because writing words is not the hard part once you get the hang of it. For me, the editing takes 80% of the overall time. Most people start as slow writers because they try to write and edit simultaneously.

Types of editing (and how they help)

With chain-of-thoughts in newer models, you don't need much prompt engineering anymore. You just need to know the right words so the LLM's thinking can go into the right embeddings.

When publishing a book, the editing steps go from the most destructive (major rewrites) to the least destructive (removing typos).

Also, as a metaexample, I’ve added the editing phases of this article here. You can see how the content changed from draft to final edit.

Let's go in that order as well:

Content Editing

Content editing improves flow, structure, and clarity. It is useful when you know what you want to say but are unsure how to connect thoughts.

It’s the most destructive, so it's better only to use it once.

Example Prompt: “You are a content editor. Improve the flow of the sentences and make the text stronger and more structured. <content>”

The AI will make many edits to make your text “make sense”, and the places where the AI misunderstood your intentions will stick out like sore thumbs. You will need to adjust them and add points that solidify your premise.

As you add (and cut) content for a second draft, it's time to move to line editing.

Line Editing

Line editing is where AI shines, especially for short texts like announcements. Use this when you know what and how you want to say something, but specific words escape you, or phrasing could be simpler.

I spend most of my time here, line editing multiple times and rewriting content until no paragraph is a waste.

Example Prompt: “Line edit this (Slack message / blog post): <content>”

Proofreading

Proofreading happens when you’ve “mastered” the copy. It’s always safe to run multiple times without fearing the AI will destroy your voice, because you will tempt yourself to write small additional bits here and there.

Example Prompt: “Fix any grammatical mistakes in the text:”

This is basically a cheap Grammarly.

As you can see, “writing” is actually two things: drafting and editing. AI struggles with the first but can greatly help with the latter. Even with better models, I don’t think we’ll completely remove the AI scent from text generated during the drafting phase.

So we as humans will need to write until we’re exhausted and don’t even want to finish what we wro—

For context: I wasn’t the “PhD scientist,” working on models. I was the guy who worked on productionizing their proof-of-concept code and turning it into something people could actually use. I worked in industries ranging from software/hardware automated testing at Motorola to small startups dealing with accessibility and education.

So here is what I've learned:

AI as a product isn’t viable: It’s either a tool or a feature

This AI hype cycle is missing the mark by building ChatGPT-like bots and “✨” buttons that perform single OpenAI API calls.

For example, Notion, Slack, and Airtable now lead with “AI” in their page titles instead of the core value they provide. Slack calls itself “AI Work Management & Productivity Tools,” but has anyone chosen Slack for its AI features?

Most of these companies seem lost on how to implement AI. A simple vector semantic search on Slack would outperform what they’ve shipped as “AI” so far.

People don’t use these products due to these “✨” AI solutions. The best AI applications work beneath the surface to empower users. Jeff Bezos comments about this (in 2016!)

You don’t see AI as a chatbot on the Amazon homepage. You see it in “demand forecasting, product search ranking, product and deals recommendations, merchandising placements, fraud detection, translations.”

That’s where AI comes in, not as “the thing” but as “the tool that gets you to the thing.”

The best use of AI is to trivialize hard problems

What if a problem that took a team of PhDs one year to solve could be solved better in four hours? That's when LLM shines:

When I worked on accessibility for nonverbal people, one of our projects aimed to make communication cards (“I want,” “Eat,” “Yes,” “No”) context-aware to allow nonverbals to express their desires faster, similar to an autocomplete.

For example, the user is home at 7 AM and taps “I want to eat” card.

The next cards should anticipate their needs (which are more likely to be breakfast items), but there are caveats: What a person typically eats for breakfast depends on their country, the type of establishment they are in (home, hotel, restaurant), the day of the week, and, of course, current personal preferences, which also change over time.

After a year of work, our team of researchers from two universities achieved a 55% rate (of the suggested options). It was a massive success at the time. We even won an award for best accessibility solution.

When ChatGPT 3.5 was released, I replicated a solution for this project and, after hacking over the weekend, got an 82% accuracy rate when running against the same test database.

There’s a reason we’re not seeing a “Startup Boom”

AI skeptics ask, “If AI is so good, why don’t we see a lot of new startups?” Ask any founder. Coding isn’t even close to the most challenging part of creating a startup.

What I do see is a boom in internal tools.

This year alone, I shipped projects that would never have been viable. As an engineering manager, spending weeks coding means neglecting the team.

The “Nice to have” bucket is when a project dies. It means there is no engineering capacity to tackle it, so it goes into the backlog limbo—until now.

Now, I can build these projects using Claude, running prompts, and reviewing the output between meetings. I see many people releasing new things that are incredibly helpful and productive, which would not have happened without Claude or Cursor.

LLMs won’t get much better, but that is okay

Like with all tools before it, we’re coming closer to the top of the S-curve for LLMs:

The last releases were unimpressive. Does anyone know a real application where ChatGPT 5 can do something that o3 could not?

The good news is that what we have is enough for most people. AI tools like KNNs are very limited but still valuable today.

This also kills the reverse FOMO: “If I wait for the technology to mature, I won’t have to deal with their earlier quirks,” is less relevant now.

But AI research is definitely not over: We will still see cheaper, faster, and open models, like those that can run on a mobile device and are as capable as ChatGPT 4o.

Ignore people trying to mystify AI

Creating AI models is hard, but working with them is simple. I put off implementing earlier AI tools because I couldn’t grasp how neural networks, sigmoids, and all that worked. Then someone said, “What are you doing? If you want to apply the technology, just use Scikit-learn.”

If you’ve never used AI for coding, install Claude Code and start using it for small tasks. That gets you 70% of AI’s current benefits without diving into prompt optimization or chain-of-thought mechanics.

Eventually, you’ll need to learn to leverage LLMs better when you hit bottlenecks. You will realize that you will still need to review code and CLI commands. You will naturally be better at prompting. You will know when and when not to use it.

AI is the new Agile: something simple, that makes you faster but has limits, yet people will position it as the solution for every problem, preaching: “Oh, you’re using (AI / Agile) wrong. In fact, it seems like what you need is even more of (AI / Agile)”

The tool has limits, especially when breaking new ground. LLMs are limited by their training data. For example, when I tried to vibecode a mod for a recently released Unity game, the AI failed to complete even a basic hook.

If You’re a Senior, You Won’t Be Replaced (It’s the Other Way Around)

Automatic railway gates replaced crossing attendants. But if those gates worked 99% of the time (or even 99.99%), would that be good enough?

LLMs are very far from being 99% accurate. They fix problems, but they tend to miss the root cause. I see many cases where the LLM suggested a fix by adding multiple lines, which an experienced engineer did by removing one.

Recognizing this requires senior-level skills, such as valuing simplicity over complexity and knowledge gained from dealing with similar bugs in the past.

This creates a problem for juniors, who, when using LLMs, will have problem-solving done for them and won’t develop this skill, hurting their code reviewing abilities. I see many companies that have stopped hiring juniors altogether.

Yes, we’re in a bubble, but that’s good for us

The Internet was a bubble in 1999, and you know the result. The internet died completely, but it was good for a while. Man, I miss the Internet.

But seriously, we are seeing great tools coming to boost productivity, a new era of AI memes, while VCs and Big Tech pay for most of them. It’s a win-win.

Also, here is my current favorite SORA video: (Warning: LOUD)

(I had to remove the video because a bug in Substack causes the space bar to play the video instead of scrolling down—sorry for the jumpscare. Here’s the Reddit link instead: https://www.reddit.com/r/SoraAi/comments/1nwcx9e/some_body_cam_footage/)

Due to time constraints, most teams run E2E nightly to avoid CI bottlenecks. However, this means bugs can slip through to production and be harder to fix because there are so many changes to isolate the root cause.

But what if we could run only the relevant E2E tests for specific code changes of a PR?

Instead of waiting hours for the entire suite, we could get the results in under 10 minutes, catch bugs before they ship, and keep our master branch always clean.

Thinking with globs

The first logical step toward running only relevant tests would be using glob patterns. We tell the system to test what changed by matching file paths.

Here's how a typical ./labeler.yml could work:

user-authentication-test: # Trigger this test
  # If any of these file changes
  - "src/auth/**/*"
  - "src/components/LoginForm.tsx"
  - "api/auth/**/*"

checkout-flow-test:
  - "src/pages/checkout/**/*"
  - "src/components/PaymentForm.tsx"
  - "api/payments/**/*"

admin-dashboard-test:
  - "src/admin/**/*"
  - "api/admin/**/*"

But globs are very limited. They require constant maintenance as the codebase evolves. Every new feature would require updating the glob patterns file.

More importantly, they cast too wide a net. A change to components/Button.tsx might need to trigger every E2E test that involves any page with a button interaction, depending on how deep the change is.

Enter Claude Code SDK

So, how can we determine which E2E tests should run for a given PR with both coverage and precision?

• We need coverage because missing a critical test could let bugs slip through to production.

• But we also need precision because running tests that will obviously pass just wastes time and resources.

The naive approach might be to dump the entire repository and changes into an LLM and ask it to figure out which tests are relevant. But this completely falls apart in practice. Repositories can easily contain millions of tokens worth of code, which makes it impossible for all AI models.

Claude Code takes a fundamentally different approach because of one key differentiator: tool calls. Instead of trying to process your entire codebase, Claude Code strategically examines specific files, searches for patterns, traces dependencies, and incrementally builds up an understanding of your changes.

So here's the hypothesis: If I see a PR, I will know which E2E tests it should run because I know the codebase. The question is: Can Claude Code replicate my human intuition by searching for it?

Let's build and find out.

Building the gatekeeper

For the E2E selection to be successful, Claude needs to know what I know: the PR modifications, the E2E tests, and the codebase structure. We need to glue all three together in a well-crafted prompt.

PR modifications

This is perhaps the easiest piece - we can leverage git to get exactly what we need. We start with the basic command:

git diff main...HEAD

This gives us the changes of a branch, but we can do much better. First, we want git to be less verbose, so we add --minimal --ignore-all-space to focus on the actual code changes rather than whitespace noise.

We also don't care about deleted files since we'll need to remove references in existing files anyway (unless we don't care about those tests), so we add --diff-filter=ACMR to exclude deleted files and focus on (A)dded, (C)opied, (M)odified, and (R)enamed files.

Finally, we need some strategic excludes because there are generally large files in PRs like package.lock that would blow up our token count. We add ':(exclude)package.lock' to keep things manageable.

Putting it all together:

git diff main...HEAD --minimal --ignore-all-space --diff-filter=ACMR -- . ':(exclude)package.lock'

The result is a clean diff showing the actual code modifications:

diff --git a/src/components/LoginForm.tsx b/src/components/LoginForm.tsx
index 1234567..abcdefg 100644
--- a/src/components/LoginForm.tsx
+++ b/src/components/LoginForm.tsx
@@ -15,7 +15,7 @@ export const LoginForm = () => {
   const handleSubmit = async (values: LoginValues) => {
-    await authService.login(values.email, values.password);
+    await authService.loginWithValidation(values.email, values.password);
   };

E2E test inventory

We could hardcode a list of test files in our prompt, but that violates the single source of truth principle. We already maintain this list for our daily benchmarks, so let's reuse it. For example, if the test configuration lives in a WebdriverIO (wdio.conf.ts) config file, we can extract it programmatically:

bun -e "import('./wdio.conf.ts').then(m => console.log(JSON.stringify(m.config.suites.liveTests, null, 2)))"

This script dynamically reads the wdio.conf.ts file and outputs our exact test suite configuration:

[
  "./test/specs/Admin/Login.spec.ts",
  "./test/specs/Admin/Vendors.spec.ts", 
  ...rest of files
]

Crafting the prompt

The prompt needs to be precise about what we want. We start by setting clear expectations:

Read the "Branch git diff" below and match it against the active "Available E2E files" below, think deep, and inspect relevant files further to decide which E2E test should run.

The key phrase here is "think deep". This tells Claude Code not to be lazy with its analysis (while spending more thinking tokens). Without it, the output was very inconsistent. I used to joke that without it, Claude runs in “engineering manager mode” by delegating the work.

Next, we set boundaries:

You should only run tests listed in "Available E2E files". If there's a reasonable chance a change could affect a test based on the actual code modifications, include it. When in doubt, **include the test**.

The "only run tests listed" constraint was added because Claude was being "too smart," finding work-in-progress spec files and scheduling them to run. We added the last piece because it is better to run more specs than leave a test out.

Getting structured output

I initially asked for JSON output, and since I didn't want Claude's judgment to be a black box, I requested two keys: the list of tests to run and an explanation. This makes it easy to benchmark whether the reasoning is sound.

I initially tried using JSON mode and asking Claude to output only JSON:

2. Output a JSON containing 2 keys, JSON only
- explanation: The explanation why you chose those E2E tests to run, be brief and concise, and use markdown format  
- tests: An array of spec file paths that should be run, like this, with no other explanation:
["./test/specs/Admin/Login.spec.ts", "./test/specs/Guest/Guest.spec.ts"]

REALLY JUST OUPUT JSON ONLY, NO YAPPIN, PLEASE DUDE JUST THE JSON

But Claude has strong internal system instructions and couldn't stop adding commentary. I initially fixed this with a regex JSON parser to remove the commentaries, but when you use regex to solve a problem, you get two problems.

But then I realized: Claude Code is used to write files, duh

So instead of fighting with JSON mode and regex, I asked:

Create a file called test-recommendations.json in the @/gatekeeper directory

Works every time!

Stitching it all together

The final pipeline combines everything with what might be the ugliest bash command known to humankind:

"e2e:gatekeeper":

cat ./gatekeeper/analysis-prompt.md \
  <(echo '\n Available E2E files: \n') \
  <(bun list-e2e-tests) \
  <(echo '\n Branch git diff: \n') \
  <(bun git-diff)

The result command is piped to Claude:

bun e2e:gatekeeper | claude -p --allowedTools "Edit Write"

We add --allowedTools "Edit Write", So it can write our test-recommendations.json file.

By the way, you should never use --skip-dangerous-permissions which gives all permissions, including Fetch. I am surprised by how many people are taught to do this. If we did add this flag, someone could write in the prompt file and instruct Claude to read our environment variables and send them to a URL using Fetch().

Since the CI runs on a PR open, not a merge, this would be similar to a “0-click” exploit.

Results: 44 minutes down to 7 minutes

I won't lie - this exceeded my expectations. We used to run all core tests, which took 44 minutes (and now it would take us more than 2 hours, since we keep adding tests). Most PRs complete E2E testing in less than 7 minutes, even for larger changes.

Even if it performed worse, it would still be an incredible success because our system has so many complexities that other types of tests (unit and integration) are nowhere near as effective as E2E.

The solution scales well because adding E2E test names consumes few tokens, and PR changes are mostly constant. Claude doesn't read all test files: it focuses on the ones with semantic naming and explores modified file patterns, which is surprisingly effective.

Did Claude catch all the edge cases? Yes, and I'm not exaggerating. Claude never missed a relevant E2E test. But it tends to run more tests than needed, which is fine - better safe than sorry.

How much does it cost? Without getting into sensitive details, the solution costs about $30 per contributor per month. Despite the steep price, it actually saves money on mobile device farm runners. And I expect these costs will drop as models become cheaper.

Overall, we're saving money, developer time, and preventing bugs that would make it to production. So it's a win-win-win!

But the truth is, junior engineers have never had it easy. When I first started applying, I had seven years of experience writing software as a hobby and still struggled to get an interview.

It wasn't until I was on the other side of the table, hiring juniors myself, that I finally understood what I had been doing wrong.

But why has the industry stopped hiring juniors?

Ironically, the current situation is worse for juniors because, just a few years ago, things were too easy for everyone. Let's rewind to the COVID era.

• Governments initiated a ZIRP (Zero-Interest-Rate Policy), meaning "safe investments" like bonds became less attractive. Investors with a lot of cash lying around needed another avenue to generate returns. As a result, tech startups became a huge target for investment.

• Also, during COVID, people needed to digitize their processes. Tools like Zoom, which were rarely used in person, suddenly became multibillion-dollar companies.

• And, with more people at home, industries like advertising, entertainment, and gaming also saw a massive infusion of cash.

These factors caused companies to hire like crazy, causing a shortage of software engineers.

With few senior engineers available, the high tide raised all the boats, and companies started poaching engineers from each other, including juniors. Recruiters were reaching even for boot camp graduates. I had a friend who was being paid and had a job offer lined up if he attended a boot camp.

However, the hiring frenzy created a long-term problem for juniors.

Everyone realized they were losing their junior engineers to other companies. Since juniors require training and would quickly get an offer elsewhere as “Senior" and leave anyway, companies stopped hiring them.

With the interest rates up, our industry is in a downturn (unless you slap “AI” in your product). I still believe juniors will make a comeback. The core principles of breaking into this industry haven't changed, and the demand for software engineers still exists.

So, how do you do that when the odds are stacked against you?

Stand out or Wash out

Even before COVID, many computer science graduates left the industry after getting their degrees. The problem was that many of them focused only on getting good grades and forgot about the actual craft of programming.

Imagine yourself as an employer reading resumes. What would set a candidate apart: their GPA or the fact that they worked on live projects that people are actually using?

I worked at a company that hired many juniors. When we reviewed their resumes, they mostly fell into two buckets, with a very imbalanced split:

• 95% of resumes just listed a bootcamp certificate or a college GPA. The rest of the space was filled with fun facts, big headshots, and flashy modern designs.

• The other 5% listed side projects, college research assistant work, public GitHub repos, Jupyter notebooks, or personal websites.

We only interviewed candidates from that 5% bucket. We knew there were capable people with high potential in the 95% pile, but like most companies, we didn't have the budget or time to interview everyone.

Getting into the 5% Bucket

It used to be that you got the job, and then you got the experience. Now, it's the other way around.

If you're in college, you're in the perfect place to build connections and find opportunities, but very few students take advantage of them. Don't wait until your final semester or when you desperately need an internship to start thinking about your career.

Unless you plan to be a researcher and pursue a master's degree, college is primarily a launchpad for your employment prospects. It's also a chance to taste different areas of computer science. Most bootcamp grads end up as mobile, front-end, or back-end engineers, but in college, you can explore other segments like working with embedded devices, firmware, or even game development.

While you're there, find other students interested in building cool stuff. Many of the biggest tech unicorns were founded by people who met in college and shared a passion. You can also pursue research opportunities with professors. You'll learn a ton, and some have valuable industry contacts who can provide strong referrals.

If you're not in college, you can start your career by doing "odd jobs" instead of only pursuing full-time employment. For example, try creating a startup. Even if the idea is bad, it's the fastest way to learn.

Freelancing is one possibility, like doing a product for that friend-of-a-friend who owns a business. This path isn't for everyone, though. After a while, you might spend more time on business and negotiation than on becoming a better coder. Those are important skills, but they can get boring if your passion is the code itself.

Another option, if you're in a country where college is cheap or free, is to go to college and apply the advice I mentioned earlier. That's what I did, I knew how to code, but wanted to learn more.

The most important thing is to keep making things and sharing them. It doesn't need to be a viable business or make a single cent. It can just be something you find helpful that might be useful to others, too.

Even when you "fail," you meet many new people. Even silly projects can lead to amazing things. I enjoy making scrapers, so back when Pokémon Go was at its peak, I built a map for my city.

One of my first users was the CTO of one of the biggest companies in my city, who encouraged me to apply to his company.

If you do enough of that, your resume will eventually cross the 95%/5% gap, and people will start calling you for interviews.

These "odd jobs" are career-defining. You will be forced to learn about optimization, caching, Redis, N+1 queries, microservices, and DevOps. You can also drop their links in your resume.

So, after a while, you will start to get interviews! Which is only half the battle.

You Will Flop Your First Interviews

Interviewing is a skill that has almost nothing to do with your actual skill as a programmer, but LeetCode-style problems aren't going away. You need to read at least the *Cracking the Coding Interview*, even if you aren't aiming for a FAANG job.

And even after you've read it, prepare to flop a few interviews. Remember that CTO who invited me to interview? I totally blew it. They asked me how I would design a database system using the Windows filesystem and folders. I basically told them the idea was silly. "Why would you create a production database with TXT files on Windows? If you need a NoSQL-style system, why not use an actual RDBMS and avoid the Windows overhead?" That's exactly how their system was built, and they didn't appreciate my candidness.

You will make mistakes in your first interviews. That's fine. It's how you learn to navigate the corporate world. You'll learn what you can say and, more importantly, what you can't. In the end, despite no one saying so, most interviewers aren't looking for the best candidate. They're more concerned with avoiding hiring the worst ones.

Once you land that first job offer and accumulate years of real-world experience, finding the next job gets easier. (It's never easy, of course, unless you have great connections).

One final piece of advice: don't focus on money too early in your career. Career growth is way more important. Joining a large enterprise might offer more job security, but a startup often gives you more opportunities to shine and get promoted faster.

With the new Flagship models coming in hot this month, like Gemini 2.5 and o1-pro, I thought it would be perfect to try those out in one category I suck most: visual design.

This is the perfect opportunity for me to replace the designer in me who is terrible with hand-eye coordination and always got bad grades in art classes because my teacher thought I was “not taking it seriously enough.” However, it is cheaper to benchmark LLMs than to go to therapy.

The Problem

I am writing a FOSS book about Engineering Management, and I have created a monstrosity of a home page below—as you can clearly see, I should get a designer.

Job to be done

We will take a screenshot and the code of the current homepage and feed it to multiple flagship AI models to ask them to make it less horrible.

The most popular reasoning models currently are Google Gemini 2.5, OpenAI o1 Pro High, xAI Grok3 Think, DeepSeek R1, and Claude 3.7 Sonnet, so we are going with those.

System Prompt

I tweaked the system prompt in some preemptive benchmarks to improve common confusion points and things I missed.

I find this prompt good enough, even for non-reasoning models:

You are taking the role of a professional website designer. Your task is to restructure the home page of a book called “The Art of Engineering Management” to be more visually attractive.

For reference, the attached image shows the website's current rendering.

The website has a Header, an Intro with a call to action, and a table of contents. Below is a skeleton of the web page with JSX components and SCSS files. The SCSS file for the table of contents is empty.

You may use any popular JavaScript libraries such as Three.js, or custom illustrations by creating SVGs if you want to. However, you may not use any CSS frameworks like Tailwind.

Your objective is to return modified files that will improve the landing page's visual appeal and attractiveness. Return only the modified files without additional explanations. Nothing should be a "TODO." You will be judged by how well your design is after the original files are overwritten.

One thing I am learning with system prompts is that if you ask LLM too much, they start failing catastrophically.

For example, it is okay to ask an LLM agent to create a unit test for a component, but if you ask: “create all missing units tests in my codebase,” it shits the bed terribly. So I am going to keep it simple.

Scorecard

Let's establish a system to evaluate the AI results. I have created a scorecard system to evaluate what we want the most:

Visual Design (50 points) - This is what we came for, so it should be half of the overall score

Interactivity (25 points) - Relates to mouse button hovers and scroll animations, basically “making it pop.”

Code quality (15 Points) - We should judge the code since having visual improvements is good and shouldn't come at the cost of code maintainability

Dark mode compatibility (10 points) - A “nice to have”: Our prompt doesn't even mention it so that we can focus on the above. If the AI messes this up, it is a quick fix.

Let's go!

I ran my current code against all the models and will share the code and the prompt when possible. Here is the original branch from which the code is used.

Deepseek R1

Let's start with the oldest one (I cannot believe I am calling an LLM released in January old). Deepseek produced a great concept with a few caveats.

The art doesn't mean much. The links with the underlines under them are fugly, and, most important, the shadows are terrible, and the hover effect of a shadow is dated.

But at least it works with dark mode!

The code is also not bad; it adds a lot of SCSS, but it is expected.

Result: Visual Design: 30 | Interactivity: 15 | Dark mode: 10 | Code Quality 10

Gemini 2.5

Google's new LLM has performed very well in many of my benchmarks. It makes the best even with the worst prompts.

This is very good. I have a few complaints about this design. I don't like the double columns of the chapters, but that is about it.

The hovering is great and presents the chapter in a very solid way!

In terms of code, it did surprisingly well, and it even fixed my bad dark mode logic, but on the other hand, it had a lot of unnecessary comments, which would not be ideal for pushing to the main as is.

Visual Design: 40 | Interactivity: 25 | Dark mode: 10 | Code Quality 10

Grok 3

Grok is the first one to disappoint me so far, but at least it made me feel better about my own design.

It repeated the book icon, “locked” the chapters as if I was selling a SaaS plan, and visually nested the cards too much.

Overall, there were some small changes, but none were positive.

It's also not stellar on the code side either, but at least I don't see any negative downsides.

Visual Design: 10 | Interactivity: 15 | Dark mode: 5 | Code Quality 10

o1 Pro High

o1 knows the key to my heart: I love blue and gradients. Maybe the fact that it did exactly how I liked it without me asking creeps me out.

I really like the header—it draws more attention than others. And instead of adding a generic SVG, it went with the best approach with the current LLM capabilities.

The chapter's menu is also not bad. I like that it starts over the banner, so it flows better.

The padding around the chapter's card could be better, and the broken white padding around the whole site might have been a mistake.

The dark mode is utterly broken.

On the code side, it is nothing bad; I could merge this as is.

Visual Design: 40 | Interactivity: 25 | Dark mode: 0 | Code Quality 15

Con… Oh wait, it's Claude Sonnet 3.7, with a steel chair!

I almost forgot Claude, to be honest. I always hear compliments on the code quality and how people prefer to use it on Cursor and Windsurf.

I am glad to see its greatness in UI design as well. That gradient is super sleek, and the subtle blue backdrop is also cool. The SVG, while not the most incredible one, was at least the most relevant.

I just didn't like that it added the famous “3 boxes with icons” that you see in every landing page template. Also, the bullet points and double columns are not visually pleasing.

Ironically, since Claude LLMs are praised for their code quality, I expected more; it tried to import a new font and tried to make the book's page flip, which clearly did not work.

Visual Design: 35 | Interactivity: 20 | Dark mode: 15 | Code Quality 5

Conclusion

The results are in:

1. Gemini 2.5: 40 + 25 + 10 + 10 = 85 points

2. o1 Pro High: 40 + 25 + 0 + 15 = 80 points

3. Claude 3.7 Sonnet: 35 + 20 + 15 + 5 = 75 points

4. Deepseek R1: 30 + 15 + 10 + 10 = 65 points

5. Grok 3: 10 + 15 + 5 + 10 = 40 points

One thing I missed in all the LLMs was improving the header text. I know that my callout is terrible because I am very bad at selling things. The system prompt even mentioned this, but all LLMs ignored it. But at least all the LLMs were successful in fixing 2 grammatical errors!

It is impressive that the AI “knows” how to better design a website without any visual aid to validate it afterward. Sure, it's not the Linear home page, and it won't win any Awwards, but in the end, I think that is the current LLM limitation. It is a blender of text absorbed by the corpus, resulting in an average of all designs worldwide.

I am also impressed by how the AI advanced. If I had tried this a few months ago, the results wouldn't be ready-to-run code. There would be just a few improvements in the code, but mostly not visual.

In the end, yes, any of those LLMs except for Grok would improve the current landing page, and I should have just applied the improvements instead of writing an article. However, I don't want an average book page. Even if I don't get paid for it, I want an excellent one.

So, for that reason, I am still getting a designer.

Speaking about emails, I am unsure when I last had a real conversation using them. Most of them just want to tell me that their company exists.

“We are updating our privacy policy.” — We both know that's not why you are emailing me. No one, not even you, cares about your privacy policy.

Email shadow advertising also takes many forms: like “How did we do? Give us feedback that we won't ever read” and “Your package has updates! (It is transit city that doesn't matter.)”

Social media

When we talk about attention, we always think about social media, which tries to get you addicted to it somehow.

If you want to test this, create a new account on Twitter / TikTok / Instagram and see what the algorithm feed "seed” posts are. They are always quasi-nudity, religion, public fights, hustlebros, or politics.

It is bad even when there is no algorithmic feed. Some years ago, Reddit's “all” page was used to entertain and have interesting news, but not anymore. If I open my Reddit /r/all page, all the posts are about <x> destroying the world, but what can I do?

None of those posts offer anything interesting or actionable. Most are rage-bait relationship stories written by AI and “clever comebacks” at a boomer politician who doesn't even know what an “ecks” is.

It is bad, even for creators

The problem is that it didn't even click with me until I started a startup, but attention is more valuable than short-term financial success.

Views and signups are cool, but it is better if the user returns to you daily and you spend a lot of time in your app. Later, you can figure out how to turn this attention into money.

There is no limit to wealth, but there is to people's lives and how much of it they spend on your app.

Using the software before it uses you

If you want to show the world something you made, you need to gather that attention—“Build and they will come” is BS for anyone who ever created something.

And if you want that attention, you need to piggyback on existing platforms to try syndicating some of their attention. But they will always get the last laugh when they rug pull you, when “your attention” is stolen back into the “platform's attention.”

For example, if I post on LinkedIn with a link attached, no one will ever see the post because LinkedIn doesn't want people to click links that get you out of the website.

If you are a YouTube creator, you know that the people subscribing is not enough—you have to ask them to hit the bell. The algorithm sometimes won't prioritize your videos to your subscribers and will prefer random videos instead.

Ironically, even Substack (this platform) does this. When I go into Substack.com, my home is not the dashboard but other unrelated people's blogs, which are nothing close to things I care about.

What can we do?

The only way out of this hell is to treat your time as a currency. “Time is money” is now on a whole new meaning. Now, your time is someone else's money.

Now, I have the same disregard for people who want to steal my time as I do for people trying to scam me out of my money.

Send me an AI cold email, and I report it as spam. Have fun getting deservedly banned by GMail. Mobile notifications are a privilege not given to any social media. Attention-seeking spam keeps coming in, but I slowly push it out.

Nowadays, I limit my social media to Subscribed Subreddits and Hacker News, but honestly, I can't resist sometimes going to Reddit's /r/all page and my local news websites and regretting it.

But like a diet, I know I might improve if I consume fewer garbage products.

Betting on the right horse

The "business need" means that advancing beyond senior relies not just on you but also on your company's circumstances. This makes getting promoted in companies that aren't growing much harder because new roles or opportunities are rarely created. And when a new senior role does open up, many internal candidates will compete for it.

This is what makes growing companies attractive. They constantly need to find new ways to capture more market share, resulting in frequent hiring, formation of new teams and initiatives, and consequently, more senior+ roles.

If you want to accelerate your career growth, consider joining a rapidly growing startup. You'll likely have greater opportunities to handle significant challenges and rapidly develop your skills. The trade-off is usually stability and possibly a lower salary.

How to know if a company is growing?

Before accepting a new role, take the time to research the company's growth trajectory. Publicly available metrics such as headcount growth, Glassdoor reviews, funding rounds, and media coverage are good indicators of a company's health.

For larger companies, you can also assess their long-term potential by reviewing customer experiences on Reddit or other forums. Generally, satisfied customers point out a positive trajectory for the company.

Being the CEO of your career

It's important not to take a passive approach to your career advancement. There's a well-known saying in Western culture: "The squeaky wheel gets the grease." In conversations with friends who mentioned deep frustrations at work, I would often ask, "What does your manager say when you discuss it?" Surprisingly often, they had never even brought it up.

Ideally, your manager should initiate conversations about your career development at least every six months. Unfortunately, many managers never discuss career development, meaning you'll likely need to be the one to bring up the discussions.

Nobody is more invested in your career than you — Be explicit about your career goals during your one-on-ones with your manager. Doing so enables more relevant feedback tailored to your desired path and helps identify skills to focus on.

If your current manager understands your goals for advancement beyond senior, they can more effectively mentor you by highlighting relevant lessons from situations that arise. Without an awareness of your aspirations, they may miss opportunities to provide valuable context or guidance.

Proactively seek opportunities with your manager to learn essential skills relevant to your desired path—whether it's interviewing candidates, running meetings effectively, strategic thinking, or directly interacting with customers.

If possible, schedule occasional skip-level meetings with your director. These conversations offer insight into the company's higher-level challenges. It also doubles as a opportunity to know and build rapport with someone who could potentially become your future boss.

Showing signs of leadership

Senior leaders generally know exactly which individual contributors (ICs) on their teams are fully prepared to step into higher-level roles if needed.

Typically, these people are highly competent senior ICs who teammates naturally approach when they need help. They're effective communicators, strong mentors, and skilled at analyzing business requirements alongside technical tradeoffs; this makes them "go-to" advisors for their colleagues.

Another invaluable trait senior leadership looks for is curiosity, essential when offering feedback constructively. For instance, if a colleague takes an unusual approach in their work, instead of calling them out directly, ask why they chose that method rather than the accepted 'ideal' approach. Framing your concern as a question demonstrates trust in their abilities and encourages them to reflect and learn.

Force multiplier

One of the most powerful actions you can take if you're seeking career growth is to position yourself as a "force multiplier." Although this concept often goes unnoticed, it significantly impacts a project's success. Force multiplier is work that once completed, enables others on the team to deliver faster and more effectively.

Whether you aspire to become a Staff-level engineer or Manager, becoming a force multiplier is an essential milestone you'll need to achieve. Staff-level leaders don't necessarily do 2–10 times the direct work of a senior individual contributor. Instead, they identify the most impactful opportunities, remove critical roadblocks, or improve infrastructure, making their teams significantly more productive.

For example, if frequent production bugs constantly stall your team's progress, implementing critical automated testing can be a significant multiplier. This ensures improvements can safely ship and saves hours of debugging later.

Another scenario could be a situation where shifting requirements frequently force team members to redo their work repeatedly. One individual who effectively communicates with stakeholders and creates comprehensive specifications before the team's efforts begin can save many hours of redundant work.

Force multiplier efforts increase everyone's efficiency. Imagine you reduce your teammates' task times by 5 minutes each, and they complete around 3 tasks per day. Across a 5-person team, you've effectively added up to **375 working days per year** to your team's productivity!

Identifying possible multipliers can be challenging since they're often highly specific to a particular context or team dynamic. However, common areas worth exploring include:

* Overly complex processes or architecture forcing team members to duplicate effort across multiple areas, introducing errors.

* Lack of robust quality checks that slow team efforts due to manual testing and debugging cycles.

* Outdated or inefficient tools complicating maintenance and hindering effective troubleshooting or productivity.

* Non-technical process inefficiencies, like unclear task definitions or ineffective planning meetings which add unnecessary overhead.

The greater the number of people impacted by your improvement, the stronger the multiplier effect.

Look proactively for gaps within your team's workflow or processes. Your manager can help you spot these bottlenecks, but as someone actively involved in day-to-day work, you're perfectly placed to notice inefficiencies firsthand. Asking your teammates directly what they like or dislike about the project can also yield valuable insights.

Getting your foot in the door

Strive to progressively take on additional responsibilities through incremental trust-building with your manager and teammates. Promotions to Staff+ roles appear to happen overnight but take months of gradual progression.

Like many others, I started as an individual contributor at a growing startup, and my responsibilities gradually shifted—from technical tasks to hiring, mentoring, planning, and client interaction. Eventually, I found myself spending most of my time guiding and leading rather than executing individually.

When it became clear that I was essentially doing the higher-level work without official recognition, I approached my senior leadership about it. Shortly afterward, I received the official recognition and title, along with greater strategic responsibilities.

Finally, even if promotion opportunities don't happen in your current company, the skills and experiences you develop make excellent talking points when interviewing elsewhere.

Whether you want to be a Staff Engineer or an Engineering Manager, this is the most crucial bar you need to clear. A staff engineer or an architect does not do 2-10x the work of senior engineers. Still, they make teams 2-10x more productive by working on areas that will improve productivity.

Force multiplier work is one that, when delivered, will allow other people to deliver faster.

If you make a specific improvement that reduces by 5 minutes of your engineer's time in each task, and if they perform about 3 tasks in a working day. In a team with 5 people, you just gave your team 375 days of work in a year. There is no 10x engineer, but there is work done by one that can have a 10x impact.

For example, a team impacted by constantly shipped bugs leads an engineer to create critical automated testing. This results in fewer bugs breaking production and speeds up every engineer on the team by reassuring them that their tickets will not break when shipping, also saving hours of debugging.

Another case would be a team that has to work back and forth frequently because new technical requirements constantly change. However, one engineer can produce excellent technical specifications by communicating effectively with stakeholders before writing the first line of code, saving the team hours of rewriting code.

The larger the number of people impacted by a positive change, the more critical the multiplier aspect is. Not every change needs to be so dramatic, but some can be even more productive, especially the ones that remove a communication overhead.

Seeking opportunities

Force multipliers are difficult to detect because they are specific to a team and codebase. But as a general rule, good potential candidates are bottlenecks and inefficiencies that drive daily aspects of the engineer's work, for example:

• Overengineered code architecture that requires the engineer to write duplicated code across codebases, with some occasional bugs.

• Quality control gaps, especially in automation, make the engineers take more time testing their tickets or shipping bugs requiring new tickets to fix.

• Outdated or reinvent-the-wheel code that introduces a lot of bugs, requires constant attention and makes it hard to find solutions to problems online due to documentation only supporting newer versions.

• There are also non-technical cases, such as proposing improvements to the shipping process, such as ticket structure and ceremonies, to avoid additional meeting overhead.

Therefore, look for gaps in your team. Your manager can help you identify those, but as someone working actively in the code, you are privileged to know how things are. Asking your fellow engineers what they dislike but have to spend their time on the project is also a good way to get that information.

I am writing an open-source book for engineering managers (EMs) seeking to enhance their skills and senior engineers aspiring to transition into an EM role. If that is your thing, check out the Github

Compare this with the OpenAI strategy. They create hype long before releasing, then casually drop it with a closed beta, with a public release right after.

It's also Google's fault that nobody follows its tools—the names change occasionally: They invested millions in Bard just to change to Gemini.

Gemini also has multiple tiers and versions with caveats, like “1.5 Pro” is better than “1.0 Ultra”. There are multiple tiers and versions, like 1.5 Pro 002—the last number being extra padded with 2 zeros means more will come to confuse everyone.

So, why should you care about them? Their AI is far from capable like ChatGPT 4o's — Gemini feels more like GPT 3.5. Well, because it excels in the needle-in-haystack problems.

ChatGPT fumbles a lot of its tokens. It theoretically gives you 16k tokens in GPT-4o, which seems like a lot, but it tends to forget the earlier tokens, the more tokens you feed it. This is probably because it might use a “rolling-window” approach, so it does not consider those earlier tokens as much as it should, but you still pay for all of them.

If you want to use Gemini tools, you must also use different websites with generic names that will probably change twice before the project gets killed. The tools are Notebook LLM and AI Studio. Not to be confused with LLM Studio, a popular FOSS tool not made by Google — I told you it was confusing.

Notebook LLM

Notebook LLM sells itself short. It claims to be a tool for studying and brainstorming ideas using your documents. They also claim they can turn documents into an AI podcast(?). At first, it seems like things a student would use to cheat help with their exams. But the tool shines by being an excellent “Google Search for your documents.”

If you feed it with large documents, it can retrieve information and make interesting critiques. For example, I am using this tool extensively for my new engineering manager book.

Having the AI do extensive critiques with sources to back them up helps me write better and not lose my agency as an author by turning things into slop. I don't want to use GenAI since it only gives generic advice from SEO-hungry websites. Using NotebookLLM means I am still in control of my writing.

AI Studio

What Notebook LLM can do for large documents, AI Studio can do for large videos. It is very useful to extract details from any media you have and seems to be the only tool to do so.

I imagine this tool being a game changer for people who use video to document things, like scouts for filming locations or real estate agents looking to write a pitch.

I sent 3 videos I made for later reference when I visited a house. I asked them to write a pitch for those videos. One thing that amazed me is that it saw a grapevine and used it for the pitch!

I've been using this tool for transcribing and annotating video evidence. My house inspection is 17 minutes long, and there was no way I was ever going back to watch the whole thing again. This transcription helps a lot to get the action points.

Something is very strange with it, though: Since the Lite mode worked so well. I thought the “Pro” model would work even better, but surprisingly, it was way worse. I don't have an explanation for this.

Another limitation from Google is that the model is fine-tuned towards safety. Some very casual videos I've uploaded triggered the warning that I might have been breaking the ToS.

It also uses the word “diverse” a lot; it probably means that, in its internal token, it should focus on diversity, but it starts using this word for everything from “diversity of rooms” to “diverse problems.” It uses this word like ChatGPT uses “delve.”

Gemini is far from beating ChatGPT and perhaps even Claude Sonnet. They are playing safe with what they have, but we should not count them out of the game. Those tools are currently free, and NotebookLLM does not train on your data. With ChatGPT, segmenting these documents/videos into tokens/images is prohibitively expensive. The thing left for us to decide is how long "free” will last…

If you are interested in the engineering manager book, I will send you the first chapters for free. These chapters will include the best and worst parts, how to succeed in the manager interview, and how the role differs in every company. If you provide feedback, I will send you the digital version of the book for free once it is released!

In a city where you have to call to order deliveries, and Google Maps won't give you their number, how can I find a place to live? Moving to a high-quality neighborhood here would cost way less than in a normal one in a big city, so searching for it would be worth it, especially for a first-time buy

By driving around, I see there are a lot of properties "for sale" announced in a sign with a number and without a real estate agency attached. I could find one by just driving it. But the city is big, with too many streets to go, and too many unmapped dead ends. It has expanded organically. And the high and low infrastructure places are just one street apart.

With Street View, you can see the “good parts” of the town. But that is still too much work and honestly, I can barely remember where I "drove" later. 

If there was a way to "cull" neighborhoods I would know where the potential home places are without driving into dangerous places. Afterward, I can narrow my search to only those places. 

The plan(TM):

1. Get all streets in the city area and points equally distributed along the road.

2. Get all the Street View pictures

3. Make GPT4 the judge of the neighborhoods.

4. Plot on a map!

I did not bother with Google Maps at first because it just had too many "false" streets, and would probably cost something. It would take years of my time to download the 30MB on JS bundle of the Google Cloud Console and navigate their menus.

So I asked ChatGPT to generate code for me using OpenStreetMaps. Not gonna lie, I didn't even want to bother reading the code it made: "I just want the .exe" I said. But the ChatGPT got horribly wrong and wouldn't bother fixing it. So like a caveman, I had to read it and fix it. 

async function getStreetNetwork(north: number, south: number, east: number, west: number): Promise<OverpassElement[]> {
    const url = `http://overpass-api.de/api/interpreter?data=[out:json];way(${south},${west},${north},${east})[highway];out geom;`;
    const response = await fetch(url);
    const data: OverpassResponse = await response.json();
    return data.elements;
}

function getPoints(street: OverpassElement, interval: number, boundingPolygon: Feature<Polygon, GeoJsonProperties>): Point[] {
    const points: Point[] = [];
    const line = turf.lineString(street.geometry.map((coord: any) => [coord.lon, coord.lat]));
    const length = turf.length(line, { units: 'meters' });

    for (let i = 1; i <= length / interval; i++) {
        const distance = i * interval
        const curPoint = turf.along(line, distance, { units: 'meters' });
        const angle = calculateAngleAtPoint(line, distance);
        if (turf.booleanPointInPolygon(curPoint, boundingPolygon)) {
            points.push([curPoint.geometry.coordinates[1], curPoint.geometry.coordinates[0], angle]);
        }
    }

    return points;
}

export default async function getStreetPoints(north: number, south: number, east: number, west: number): Promise<Point[]> {
    const points: Point[] = [];
    const interval = 70;  // meters

    // Define the coordinates of the bounding box
    const coordinates = [
        [
            [west, north], // Top-left corner
            [east, north], // Top-right corner
            [east, south], // Bottom-right corner
            [west, south], // Bottom-left corner
            [west, north]  // Closing the polygon by returning to the starting point
        ]
    ];

    // Create the polygon using Turf.js
    const boundingPolygon = turf.polygon(coordinates);

    const streets = await getStreetNetwork(north, south, east, west);

    for (const street of streets) {
            points.push(...getPoints(street, interval, boundingPolygon));
        
    }

    // Filter near 40m
    const filteredPoints = filterCloseGPSPoints(points, 40);

    for (const point of filteredPoints) {
        console.log(`[${point[0]}, ${point[1]}, ${point[2]}],`);
    }
    return filteredPoints;
}

For the roads I marked the points at every 70m, in imperial units that is 0.76 football fields, I had a “bug” that every point at the intersection on the road was marked. But it was due that the intersection is the "beginning of the road". Duh!

2. Getting the Street View photos

It is super duper expensive now to use Google APIs. And Playwright is easier than reading Google documentation, so I am cashing the favor for contributing long enough to Google Maps. 

One huge tip is to make the dimension of the image a multiple of 512 for OpenAI, a 512x512 image costs 4x less than a 513 x 513 one. It says in the documentation that it splits the image in 512 x 512 chunks, or 85 tokens each, we do 1024x512, so it is wide and only 2 tiles!

// Generate 2 angles to capture
const points = pointCloud.map(([lat, lng, deg]) => {
    return [
        [lat, lng, deg % 360],
        [lat, lng, (deg + 180) % 360],
    ];
}).flat();

// Set up Playwright Chromium browser
const browser = await chromium.launch({headless: false});

// So we spend only 2 tiles!
const context = await browser.newContext({viewport: {width: 1024, height: 512}});

const promises = points.map(([lat, lng, deg]) => limit(() => getPhoto(context, lat, lng, deg)));
await Promise.all(promises);


// Screenshot code

const filename = `screenshots/screenshot_${lat}=${lng}=${deg}`.replace(/\./g, '_') + ".png";
if (fs.existsSync(filename)) {
    console.log(`Skipping ${filename}.png`);
    return fs.readFileSync(filename);
}

const page = await context.newPage();

const url = `http://maps.google.com/maps?q=&layer=c&cbll=${lat},${lng}&cbp=11,${deg},0,0,0`;
console.log(`Navigating to: ${url}`);
await page.goto(url);

// Wait for the page to load (wasteful, I know)
await page.waitForTimeout(5000);

// Run the script to delete children except for 'id-scene'
await page.evaluate(() => {
    const container = document.querySelector('.id-content-container');
    if (container) {
        container.querySelectorAll(':scope > :not(.id-scene)').forEach(child => child.remove());
    }
});

// Take the screenshot
const screenshot = await page.screenshot({ path: filename, fullPage: true, type: 'png' });

After the images were downloaded I spent hours promptly writing and got mad that the first short draft I made was the best one. GPT4 gets biased if you give too many details, like if it is Brazil mentioned, it starts to sugarcoat how bad the street is:

If you add what factors to consider to the prompt, it will hyperfocus on looking for a positive aspect in the input. A street with an abandoned lot was given credit for its “greenery”. Probably ChatGPT has instructions for it to be positive with what you give in the prompt and this case is not ideal.

Another good surprise is that the new structured output is awesome, you don’t need to YELL at the AI to not over-explain things, or safeguard it, and it spared a lot of time not calibrating the prompt.

const images: ChatCompletionContentPartImage[] = photosPath.map(file => {
    const image = typeof file === "string" ? 
        convertImageToBase64(path.join(SCREENSHOT_DIR, file)) : 
        file.toString("base64");
    return {
        type: "image_url",
        image_url: {
            url: `data:image/jpeg;base64,${image}`,
        }
    };
});

const completion = await openai.beta.chat.completions.parse({
    model: "gpt-4o-2024-08-06",
    messages: [
        {
            role: "system",
            content: `
Your task involves receiving street view images and classifying the neighborhood infrastructure on a scale from 0 to 5. 
You will evaluate the neighborhood based on the average quality of its housing and pricing and translate that to a level. 
A low level (0) might represent a street with dirt roads and unfinished houses with exposed bricks.
A high level (5) would indicate a well-developed area with asphalt roads and high-quality houses.
`
        },
        {
            role: "user", content: images,
        },
    ],
    response_format: {
        type: "json_schema",
        json_schema: schema
    },
});

if (completion.choices[0].message.refusal) {
    throw new Error(`API refused to rate pointPhotos: ${lat}, ${lng}`);
}

return completion.choices[0].message.parsed as any as Response;

3. Mo data mo problems

Last thing, there are some loose spots, where there are some few good houses in an overall bad street. Those are not ideal candidates, I used something similar to KNN to remove that "noise". Yes, I made a O(n**2) code since my time is worth more than the energy that my Mac will use.

// Function to find the 4 nearest neighbors of a given point
function findNearestNeighbors(point: { lat: number, lng: number }, points: any[], k: number = 4): any[] {
  return points
    .map(p => ({
      ...p,
      distance: calculateDistance(point, p)
    }))
    .sort((a, b) => a.distance - b.distance)
    .slice(1, k + 1); // Ignoring the first one because it will be the point itself
}

// Function to modify points based on nearest neighbors
function adjustRatings(ratings: any[]): any[] {
  return ratings.filter(point => point.rating != -1).map(point => {
    const nearestNeighbors = findNearestNeighbors(point, ratings);
    const maxNeighborRating = Math.max(...nearestNeighbors.map(p => p.rating));

    if (nearestNeighbors.every(neighbor => neighbor.rating !== point.rating) && point.rating > maxNeighborRating) {
      return { ...point, rating: maxNeighborRating };
    }

    return point;
  });
}

Overall it was great enough, looking at the rating and the images in a vacuum like the AI did I would rate them the same as well. There were no disagreements between me and AI, which would invalidate the whole thing.

I also removed the beginning of the street in my code and removed nearby points to save some tokens.

Here is the final thing, using QGIS and TIM interpolation among points:

Insights

The good news was every place got at least 1, despite telling it to rate from 0-5, there were no legit zeros given. Even on muddy roads with large vegetation. But that was a good thing, it independently decided to reserve 0 to "not found" images and I could easily filter those out.

The bad news was that no place got a 5, the one that got a 4 was removed by the KNN, which is good for the quality of the rating but bad news for the city overall. I think a 4-5 would be a posh American suburbia with an HOA and lots of grass lawns and a large asphalt road with no overhead wires, and we have that here.

A surprise was how consistent it was to judge according to its own parameters, all the prompting was done in isolation, but remained the same across nearby road spots. There were rarely sudden jumps between 1 to 3 for example.

Another surprise was that the AI loved parks, it could have the saddest playground ever, if there was a park it got at least a 1-point bump, but that makes sense, a park is a “permanent view" of some sort.

What's next

Now I need to visit those places and drive around to see what is there to sell, next steps might be getting a 360 camera, putting it on the roof of the car, a lá street view, and taking pictures and looking for "for sale signs". Stay tuned!

Subscribe now

Share