July, 2021 - JS
I was asked to summarize what I thought were the core engineering values I brought to my teams, this is what I came up with.
Core Values
- The main goal of engineering is to provide solutions for users.
- Engineering delivers people to Product; Product delivers value to users.
- Stable teams perform better over time, the longer people work together, the more they build trust and momentum. That being said, sometimes people need to move - for variety, learning, priorities, or culture reasons. Aim for low churn, but don’t keep things too static.
- Good engineering is data-driven, we must measure things to improve them. “The only difference between science and screwing around is writing it down” - Adam Savage.
- Engineers expect to be paid competitively on skill, and really good engineers have to get paid really well. But there has to be a seniority structure, aka the “engineering ladder”. Eliminate inconsistencies or problems with pay and titles. Make sure that if anyone leaked their salary, nobody would get hurt, because it’s all set up fairly and within market rates.
On Building Things
- Product is about the “why” and “what”, engineering is about the “how” and “when”
- Code is not an asset, it’s a cost. Every line of code adds debt. The true price of software is maintenance, not development.
- Build vs Buy: understand the REAL costs, maintenance of DIY, speed, licenses, standards.
- Encourage transparency and openness across the board: all code, documentation, design, research and plans are open to everyone in the company. Feedback and review is encouraged, contributions are welcome.
On How We Build
- Understanding the Agile Principles - the hype from reality
- Talking to customers directly (even engineers!)
- Responding to change (and dangers of over-planning)
- Interactions over processes/tools (talk and pair, vs. making tickets)
- “Working software” over specs, never break the build, “always be deploying”
- Faster iterations beat slower and longer cycles. Less risk, faster learnings, more chance for improvements. People think they save time with longer sprints, but they get less practice in their plannings, demos, and retrospectives. Keep rituals short and efficient, aim to have them frequently.
- Engineering is a team sport, not a solo competition. Teams build great things, not individual developers. The company as a whole and the platform must work productively together, aimed at building software.
- Standards lead to efficiency. Doing things in multiple ways costs code. Easy integration is a goal. Build with legos, not molecules.
- We never get things right the first time. This is a celebrated truth of good engineering. Build scale models, prototype with found objects, then build it right.
- Open source and inner source - use pull requests (PRs), automated test/deployment, anyone can raise issues, and healthy debate is encouraged. PRs build consensus on standards and highlight differences. PRs reduce bugs and incidents.
- Invest in good documentation. Navigating and documenting code is a huge cost. Optimize code development for understandability and changeability, over other concerns like re-usability, time-to-market and performance.
Ownership
- Teams take care of their services and APIs (not individuals)
- Every codebase needs more than an “owner”, a tribe is needed
On Teams
- Teams need to be autonomous and empowered. Product, Engineering Design and whoever else is needed. The best teams have strong ownership, a clear mission, and permission to do what’s necessary.
- Structure teams around "domains" (like billing, accounts, search/explore) rather than tech (mobile, web, backend, etc) or projects or user journey steps. Conway's Law teaches us that the team setup will define the architecture.
- Every engineering team needs a Product Manager (PM) and a sponsor from management. Without a link to product and business, the team is at risk to be alienated or taken off track. Engineers should not take product decisions alone, it erodes trust.
- Every team and PM always wants more resources. You must challenge them to be efficient, and focused. Adding more people does not usually equal more progress, and only makes sense if you’re really ready to scale.
- Engineers need to be involved in product decisions, talking to customers, and contributing to solutions. Avoid the “design it and give it to the devs” pattern. Engagement is much higher for teams who are fully engaged in the product process.
- Avoid multiple parallel tracks on different topics in a team, try to focus everyone in the same solution space at once.
- Encourage different approaches for building: multiple wireframes/spikes/demos often lead to better results than an “MVP” release. Explore the problem space with real data when possible.
The Engineering Organization
- Avoid “handover trains” (PM->specs->design->review->engineering). Make teams build things iteratively, together.
- Engineers are like bees - when they are allowed to inter-mingle, they tend to pollinate good ideas around. Use hackdays, guilds, trainings, and conferences here to help.
- All teams should align their sprints (and production increments) to start and end at the same time. This saves enormous costs on coordination.
- The most powerful rituals in a healthy company are all-hands planning (the entire company plans together), and public demos (all work is shown to all employees), and the management thinks so too
- Everyone demos (even failures!) regularly
- All source is open, anyone can review and make a PR
- Key Metrics (KPIs) in the team are visible and everyone knows where we stand
- We value sharing knowledge, collaborative documentation, best practices, RFCs. We frown on opinionated manifestos and prescriptive preaching.
Deployment and Quality
- Announce all major changes and releases, along with metrics/KPIs
- “Swarming behaviour” - when there’s a problem, the whole team takes care.
- We remove restrictions: access, limits on deployment times, etc
- Use blameless post-mortems to handle your failures, it's invaluable for learning and trust-building.
- Trust means fast releases and rollbacks
- Operations: you build it, you run it. But having a good on-call and SRE team helps.
- Feature flags add speed and agility, avoid “big releases”
- Test everything: the code as well as the product performance (A/B, experiments, real-time feedback, etc)
- use a Release Checklist (SEO, security, tracking, etc)
- Be brutal: replace code when better solutions are clear
- A mature org is able to place devops and QA directly into teams, rather than having them separate
- Use SLAs/SLOs and error budgets rather than rules for deployments.
Pride
- “We are not our code”, it’s the value we deliver, not how clever or perfect our code is
- Pride in engineering is how simple or easy our solutions are, the complexity we eliminate
- Failure is experience, it makes us better
- We value time to delivery over performance and reusability
- We share our successes by recognizing our team mates
Leadership
- Engineering leaders set the tone for what’s acceptable. Are we ok with no tests? With errors in production? With passwords stored in git? The “broken window” policy sets the culture for how engineers behave.
- Good leaders attend their team's demos, read the things they write, and participate in reviews. They act as part of the team and aim to serve and support.
- The number one failure most CEOs make: relying on reports and middle management for decisions, instead of being "on the shop floor" regularly with the staff and customers.
- Engineers show leadership not just by managing people/projects - it's also in how they review code, champion quality, take initiative, improve things, and help others.
- Top management needs to encourage an environment where engineers are safe to fail and share crazy ideas without criticism.
- Learning and ideas about product and design are encouraged, engineers should be seen as enablers, not as “the working class”. Make everyone care about product and aim to improve it, especially engineers.