Engineering Team Organization Through Boundaries

As engineering organizations grow beyond their initial startup phase, they face an inflection point. The intimate collaboration and fluid communication that worked wonderfully with a handful of engineers starts breaking down as the team expands beyond 20 people. What was once a tight-knit group that could swarm on problems together becomes a complex organism requiring thoughtful structure to function effectively.

In this chapter, we'll explore how to organize engineering teams as they scale, with a particular focus on the concept of boundaries. We'll examine the traditional approaches of product vertical and functional team structures, their tradeoffs, and ultimately arrive at a more nuanced perspective that uses boundaries as the foundation for organizational design.

The Traditional Approaches: Product vs. Functional

Let's start by examining the two conventional ways of structuring engineering teams: product verticals and functional specializations.

Product verticals organize teams around user-facing features or business capabilities. For example, you might have separate teams for user authentication, payment processing, and content management. Each team owns their vertical end-to-end, from frontend to backend to infrastructure.

Functional teams, on the other hand, organize around technical specialties. You might have dedicated frontend, backend, infrastructure, and machine learning teams. These teams work across different product features, bringing their specialized expertise to bear on various projects.

Each approach has its strengths and weaknesses:

Product Vertical based Teams:

• Advantages:
  • Teams have clear ownership of specific user experiences
  • Faster decision-making within their domain
  • Better alignment with business outcomes
  • Reduced coordination overhead for shipping features
• Challenges:
  • Potential duplication of technical solutions
  • Risk of inconsistent practices across teams
  • May struggle with platform-wide technical initiatives
  • Can be harder to maintain technical excellence in specific domains

Functional Teams:

• Advantages:
  • Strong technical excellence in specific domains
  • Consistent practices within each specialty
  • Efficient sharing of specialized knowledge
  • Better leverage of technical expertise
• Challenges:
  • Higher coordination costs for shipping features
  • Slower decision-making due to dependencies
  • Risk of becoming disconnected from business outcomes
  • Can create bottlenecks when multiple product initiatives need the same functional team

As organizations scale, I've observed that product-aligned teams generally serve them better. The ability to move quickly and independently on business initiatives usually outweighs the benefits of functional specialization. However, this isn't a binary choice – and that's where boundaries come in.

Boundaries: A More Nuanced Approach

Rather than choosing strictly between product and functional organization, we can take a step back and look at our system architecture as a whole. Boundaries represent logical separations in our problem domain that can help us organize teams more effectively.

Think of boundaries as the natural fault lines in your system – places where different parts of your business or technical domain naturally separate. These boundaries might align with product verticals (like "user management" or "content delivery") or functional concerns (like "data platform" or "authentication"), but they're defined by the inherent structure of your problem space rather than organizational convenience.

The benefits of organizing around well-defined boundaries are numerous:

First, boundaries create clear ownership. When an incident occurs or a new feature is needed, there's no confusion about which team needs to respond. This clarity reduces the coordination overhead that often plagues growing organizations.

Second, boundaries enable autonomy. Teams can work independently within their boundaries, making local decisions without requiring consensus from the entire organization. This autonomy speeds up development and helps teams stay motivated by giving them real control over their domain.

Third, boundaries reduce cognitive load. New team members don't need to understand the entire system to be effective – they can focus on their boundary and treat other boundaries as black boxes with well-defined interfaces. This makes onboarding faster and more effective.

Fourth, boundaries enable incremental improvement. Rather than requiring massive, risky platform-wide changes, teams can innovate within their boundaries. These smaller, more focused improvements are easier to implement and validate.

Finally, boundaries reduce analysis paralysis. When teams have a clear scope of responsibility, they can make decisions more quickly without getting bogged down in system-wide implications.

Principles for Effective Boundaries

To realize these benefits, boundaries need to be designed thoughtfully. Here are some key principles to follow:

Independence is crucial. Boundaries should be as isolated as possible, with minimal dependencies on other boundaries. This isolation enables teams to work independently and reduces the coordination overhead that can slow down development.

Skills alignment matters. While boundaries aren't purely about team organization, they should group together work that requires similar skills. This alignment makes it easier to staff teams and enables knowledge sharing within boundaries.

Ownership should be clear. Each boundary should be owned by no more than one or two teams. When ownership is spread across more teams, coordination costs increase and accountability becomes unclear.

Communication between boundaries should happen through well-defined contracts. Teams should be able to treat other boundaries as black boxes, interacting with them through stable interfaces rather than needing to understand their internal workings. This enables faster research and development, innovation and general delivery speed. One method of creating contracts is via APIs. Whether it's GRPC, GraphQL, REST, logically separated services, or SDKs, it doesn't matter. The point is that systems and people don't need to know the fine grained details in order to leverage other systems for speed.

It's important to note that boundaries are not the same as implementation details. For example, while microservices might be one way to implement boundaries, the boundaries themselves exist at a higher level of abstraction. The same boundaries could be implemented within a monolith, as separate services, or through other architectural approaches.

Think of boundaries as the lines we draw on our problem space to make it manageable. The implementation details of how we enforce those boundaries are important but secondary considerations.

Each boundary should have a mission statement. Boundaries should have mission statements or goals that they're shooting for that's rooted in business need. The next level detail would be having clear metrics that the boundary owners will be optimizing for.

Boundary Examples

Indonesian Health App Fita

Whilst working for BCG, I built out a couple of corporate ventures in Indonesia as the CTO. Hired about 30 software engineers, including the replacement CTO and they took Indonesia by storm to make the country healthier. A few months after I left, there were 1M active users, it was the #1 trending health app and people lost 8kg in 12 weeks on average. We'll use this app as an example for boundaries. This is what the app looks like as a snapshot of the play store:

The main features of the app for the purposes of the boundary excerise include:

• healthy meal and exercise courses created by health and fitness experts
• courses can have "likes"
• people can book text chat sessions with certified coaches to help them get healthier
• people can enroll in courses. Courses can span days to weeks.
• there's a diary where people can log what they ate or did
• there's some biometrics/steps tracking
• people can earn points and redeem them for goods and services
• there's a bunch more but let's ignore it for the purposes of this exercise

Example Boundaries for Fita

This is a philosophical exercise in potential boundaries, not the exact ones that are in Fita. You may slice it a bit differently depending skillsets and how the organization functions but going through the exercise is worthwhile no matter how it's cut. As long as there is organizational agreement on where boundaries lay, the rest will sort itself out.

In addition to the boundaries, you can start having robust conversations on which teams own which boundaries and where the communication lines need to be a bit more tight.

In this case, the health content team might want to speak closely to the social engagement team to make sure that course information is effective, valued, liked and there's plenty of interaction in the core product offering.

Also notice that 2 out of 6 boundaries are functional. You want to make sure to keep that balance and not overdo functional vs product. Though you could slice this more heavily by product if you had good guard rails and automations in place (see below).

By having clear boundaries, issue triaging, ownership etc. become significantly easier to identify. It also enables further innovation as you can call the internal contracts of any of the boundaries to spin up new verticals without having to understand how it works under the hood.

Implementing Boundaries: Technical Guardrails for Team Autonomy

When we establish boundaries in our engineering organization, we're not just drawing lines on an org chart – we're creating spaces where teams can innovate and move quickly while maintaining the overall integrity of our system. Think of it like city planning: we want each neighborhood to have its own character and local governance, but we also need infrastructure and rules that make the whole city work together harmoniously.

To achieve this balance between autonomy and consistency, we need well-designed guardrails. Let's explore the key components that make this possible.

Code Stewardship and Certification

One of the most effective ways to maintain quality within boundaries is through a formal stewardship program. Think of stewards as master craftspeople who deeply understand specific aspects of your system. They're not gatekeepers who slow things down, but rather mentors who help teams succeed.

A CODEOWNERS file is often the technical implementation of this concept, but the real value comes from the human element. For example, if you have a shared authentication service, you might have several certified authentication experts across different teams. These experts stay current with security best practices, attend relevant conferences, and help review critical authentication-related changes.

The certification process itself should be collaborative and growth-oriented. Instead of creating bottlenecks, certified experts should focus on teaching and enabling others. They might run workshops, create documentation, or pair program with team members who want to learn more about their specialty.

Self-Service Infrastructure

A robust CI/CD framework is like giving teams their own workshop with all the tools they need. Rather than waiting for a central team to set up their pipeline, teams should be able to configure their own builds, tests, and deployments within established patterns.

Consider creating a pipeline-as-code template system where teams can start with proven patterns and customize them for their needs. For instance, a team might start with a standard microservice pipeline template but add specialized performance tests specific to their boundary's requirements.

The key is to make the right thing the easy thing. Your CI/CD framework should make it simple to implement logging, monitoring, security scans, and other cross-cutting concerns. Teams shouldn't have to think about how to implement these basics – they should be able to focus on their unique domain challenges.

Enterprise Architecture Alignment

Enterprise architecture in a boundary-based organization is less about control and more about enabling consistent evolution. Rather than acting as a checkpoint that teams must pass through, enterprise architecture should provide clear patterns and guidelines that teams can follow autonomously.

For example, instead of requiring teams to get approval for every technology choice, create a technology radar that shows preferred, accepted, and deprecated technologies. Teams can then make informed decisions within these guidelines, coming to architecture review only for exceptions or novel cases.

Document your architectural decisions and their rationales in a searchable format. When teams understand why certain patterns are preferred, they're more likely to make choices that align with the broader organizational direction.

Data Governance and Privacy

In today's regulatory environment, data governance isn't optional – it's essential. Each boundary needs to understand and implement appropriate data handling practices. This might include:

• Clear data classification standards that help teams understand how to handle different types of information
• Automated scanning tools that can identify potential privacy issues in code or configurations
• Standard patterns for data encryption, retention, and deletion that teams can easily implement
• Regular training and updates on privacy requirements and best practices

Teams should be able to answer questions like "What personal data does my boundary handle?" and "How do we ensure GDPR compliance?" without having to consult a central authority for every decision.

Non-Functional Requirements

Performance, accessibility, security, and other non-functional requirements shouldn't be afterthoughts – they should be built into the development process. Create automated checks and tools that help teams maintain these standards:

• Accessibility testing in the CI pipeline
• Performance budgets that automatically fail builds if they're exceeded
• Security scanning integrated into code review processes
• Load testing as part of the release process

The goal is to make these requirements visible and actionable without requiring teams to become experts in every area. Tools should provide clear feedback and actionable suggestions for improvement.

Developer Productivity Tools

Time spent on boilerplate code or repetitive tasks is time not spent on innovation within a team's boundary. Invest in tools that accelerate development while maintaining consistency:

• Code generators for common patterns (new services, API endpoints, database migrations)
• Development environment setup scripts that ensure consistent tooling
• Shared component libraries that implement common UI patterns
• API client generators that create type-safe interfaces between boundaries

These tools should encode best practices while remaining flexible enough to accommodate legitimate boundary-specific needs.

Security at the Boundaries

While each team needs to think about security, having strong controls at your system's entry and exit points is crucial. An API gateway acts as both your first line of defense against external threats and your last line of defense for internal services.

Your gateway should handle:

• Authentication and authorization
• Rate limiting and DDoS protection
• Request validation and sanitization
• Response filtering and security headers
• Logging and monitoring of all boundary-crossing traffic

This centralized control point lets teams focus on their domain-specific security concerns while ensuring consistent protection at the system level.

Building a Culture of Enablement

The success of these guardrails depends on how they're perceived and implemented. They should feel less like restrictions and more like tools that help teams succeed. Some ways to build this culture:

• Regularly review and update guardrails based on team feedback
• Celebrate teams that innovate while staying within the guardrails
• Share success stories and lessons learned across boundaries
• Provide clear escalation paths when guardrails need to be adjusted

Remember, the goal of these guardrails isn't to constrain teams – it's to give them the confidence to move quickly within their boundaries while maintaining the integrity of the overall system.

Finding Your Balance

While I've advocated for a boundary-based approach, there's no one-size-fits-all solution. The right balance between product and functional organization will depend on your specific context – your team size, business domain, technical challenges, and growth trajectory all play a role.

That said, as organizations scale beyond their initial startup phase, I've found that leaning towards product-aligned boundaries typically works better. The ability to move quickly and independently on business initiatives usually outweighs the benefits of pure functional specialization.

The key is to identify your natural boundaries and organize around them in a way that enables teams to work independently while still maintaining technical excellence. Start by mapping out your system's natural fault lines, looking for places where different parts of your business or technical domain naturally separate.

Remember that this isn't a one-time exercise. As your organization grows and your problem space evolves, your boundaries may need to shift. Stay flexible and be willing to adjust your organization as circumstances change.

Conclusion

Organizing engineering teams is one of the most challenging aspects of scaling a software organization. While the traditional dichotomy between product and functional organization offers useful insights, thinking in terms of boundaries provides a more nuanced and effective approach.

By identifying and respecting the natural boundaries in your system, you can create an organization that combines the best aspects of both product and functional organization. Teams can work independently within their boundaries while maintaining technical excellence and alignment with business goals.

The key is to approach this thoughtfully, with a clear understanding of your specific context and challenges. Take the time to identify your natural boundaries, and organize around them in a way that enables both independence and collaboration. Your teams – and your products – will be better for it.