One Project To Rule Them All

The Trilogy – Part 1

In this 3-part series showcased on Microsoft’s Premier Developer Blog, we will discuss the business value of having a single organization, single project approach in your Azure DevOps journey.

Why a single project approach you ask? Usually, organizations or enterprises start their Azure DevOps journey in one of two ways:

1. A migration from pre-existing on-premise Azure DevOps Server (aka Team Foundation Server) project collection(s). This is usually done using High Fidelity tools such as the TFS Migrator Tool.
2. Organically: meaning the organization or enterprise initially creates an Azure DevOps Organization with a project in it. Sooner or later, the need is felt to add more projects to this initial ADO organization or even create supplementary ADO organizations to house additional projects.

In any case, the organization or enterprise may come to the realization that these scattered projects in multiple ADO organizations have become unwieldly and difficult to manage. A side note: henceforth, when we mention the word enterprise, we mean “organization or enterprise” but for the sake of brevity and to avoid confusion with an Azure DevOps organization, we may have a preference for the former expression. Common challenges are:

• Dealing with a fragmented view of work within the enterprise
• Planning vertically from the portfolio (or leadership) level down to the level of the project or feature teams
• Conversely, obtaining a wider context of work by going up to the portfolio level
• Planning horizontally across multiple project or feature teams (e.g. viewing Gantt charts of all the feature teams on a common timeline and cadence)
• Managing security enterprise-wide
• Sharing source code, pipeline definitions, test plans, and artifacts across the enterprise
• Facilitating mobility and fluidity of individual team members from one team to another
• Acquiring Business Intelligence from the multitude of scattered or even siloed projects
• Delivering OKR‘s, KPI‘s, Metrics or Reports enterprise-wide

The above challenges can be dealt with using a single organization, single project strategy. The “One Project To Rule Them All” idea is certainly not new. Indeed, it’s mentioned by Martin Hinshelwood in this July 2012 blog post. More recently, in May 2018, Colin Dembovsky wrote an interesting blog post in the matter that I encourage you to read. Finally, the Microsoft Documentation on “When to add another team project” explicitly states:

In general, we recommend that you use a single project to support your organization or enterprise.

Unfortunately, many enterprises have multiple organizations and projects in their portfolio and merging them into a single project can seem a daunting task as can be seen by this request. While there are tools out there that can help, there is no “Single Tool To Rule Them All” if I may express myself this way. Nonetheless, we have the following at our disposal:

1. Azure DevOps Services REST API
2. Naked Agility’s Azure DevOps Migration Tools
3. Azure DevOps Process Tools which is a wrapper for Microsoft’s process migrator

The reality is that it’s a large endeavour to migrate multiple projects into a single one with minimal disruption. In addition, different enterprises or organizations have different needs. For instance, a consulting firm housing projects for its customers and inviting them into their organization would require more isolation in order to prevent customer A from accessing critical or sensitive information of customer B such as source code.

Apart from cloning a project from one organization to another, all other project migrations such as merging multiple projects together are by their very nature not high fidelity. Typically, one migrates a project into a single parent or host project by moving its work items under a fenced Area of the parent project. Similarly, the migrated project’s pipelines are placed within some isolated folder of the host project. This allows us to migrate multiple leaf projects within a common host infrastructure or trunk reflecting the enterprise’s portfolio and/or program structure.

Let’s illustrate the above with a concrete example. I started with an Azure DevOps organization called devopsabcs containing 29 projects based on the Scrum process. A good portion of these projects were part of my training portfolio to pass the AZ-203 Exam: Developing Solutions for Microsoft Azure. Other projects are part of other portfolios such as my Research portfolio containing various R&D projects. The following abridged json describes a subset of the projects to merge:

{
	"TargetOrganizationName": "agileatscale",
	"TargetProjectName": "OneProject07",
	"TargetPAT": "ZZZtargetOrgPAT2ZZZ",
	"CoreRootOutputFolder": "c:\\templates",
	"RootPathToConsoleApplications": "c:\\src\\devopsabcs\\migrationtool",
	"SimplifiedProjects": [
		{
			"IsClone": false,
			"SourceOrganizationName": "devopsabcs",
			"SourceProjectName": "ApplicationInsights",
			"SourcePAT": "ZZZsourceOrgPATZZZ",
			"AreaPrefix": "Training\\CertAZ203",
			"TeamPrefix": "AI_"
		},
		{
			"IsClone": false,
			"SourceOrganizationName": "devopsabcs",
			"SourceProjectName": "CosmosDB",
			"SourcePAT": "ZZZsourceOrgPATZZZ",
			"AreaPrefix": "Training\\CertAZ203",
			"TeamPrefix": "CDB_"
		},
		{
			"IsClone": false,
			"SourceOrganizationName": "devopsabcs",
			"SourceProjectName": "DevOpsDemoGenerator",
			"SourcePAT": "ZZZsourceOrgPATZZZ",
			"AreaPrefix": "Research\\Migrations",
			"TeamPrefix": "DG_"
		},
		{
			"IsClone": false,
			"SourceOrganizationName": "devopsabcs",
			"SourceProjectName": "SomeFinTechCo-Hackfest-PoC",
			"SourcePAT": "ZZZsourceOrgPATZZZ",
			"AreaPrefix": "Customer\\FinTech",
			"TeamPrefix": "FIN_"
		},
		{
			"IsClone": false,
			"SourceOrganizationName": "devopsabcs",
			"SourceProjectName": "hol",
			"SourcePAT": "ZZZsourceOrgPATZZZ",
			"AreaPrefix": "Workshops\\DevOps",
			"TeamPrefix": "HOL_"
		},
		{
			"IsClone": false,
			"SourceOrganizationName": "devopsabcs",
			"SourceProjectName": "ManagedIdentities",
			"SourcePAT": "ZZZsourceOrgPATZZZ",
			"AreaPrefix": "Training\\CertAZ203",
			"TeamPrefix": "MI_"
		},
		{
			"IsClone": false,
			"SourceOrganizationName": "devopsabcs",
			"SourceProjectName": "MigrationTool",
			"SourcePAT": "ZZZsourceOrgPATZZZ",
			"AreaPrefix": "Research\\Migrations",
			"TeamPrefix": "MT_"
		},
		{
			"IsClone": false,
			"SourceOrganizationName": "devopsabcs",
			"SourceProjectName": "PartsUnlimited",
			"SourcePAT": "ZZZsourceOrgPATZZZ",
			"AreaPrefix": "Demo\\Migrations",
			"TeamPrefix": "PU_"
		},
		{
			"IsClone": false,
			"SourceOrganizationName": "devopsabcs",
			"SourceProjectName": "PowerBI",
			"SourcePAT": "ZZZsourceOrgPATZZZ",
			"AreaPrefix": "Research\\BusinessIntelligence",
			"TeamPrefix": "PBI_"
		},
		{
			"IsClone": false,
			"SourceOrganizationName": "devopsabcs",
			"SourceProjectName": "SmartHotel360",
			"SourcePAT": "ZZZsourceOrgPATZZZ",
			"AreaPrefix": "Demo\\Migrations",
			"TeamPrefix": "SH_"
		},
		{
			"IsClone": false,
			"SourceOrganizationName": "devopsabcs",
			"SourceProjectName": "SpecFlow",
			"SourcePAT": "ZZZsourceOrgPATZZZ",
			"AreaPrefix": "Research\\Testing",
			"TeamPrefix": "SF_"
		},
		{
			"IsClone": false,
			"SourceOrganizationName": "devopsabcs",
			"SourceProjectName": "spring-music-devops",
			"SourcePAT": "ZZZsourceOrgPATZZZ",
			"AreaPrefix": "Research\\OSS",
			"TeamPrefix": "SPR_"
		},
		{
			"IsClone": false,
			"SourceOrganizationName": "devopsabcs",
			"SourceProjectName": "VstsWorkItemMigrator",
			"SourcePAT": "ZZZsourceOrgPATZZZ",
			"AreaPrefix": "Research\\Migrations",
			"TeamPrefix": "WIT_"
		},
		{
			"IsClone": false,
			"SourceOrganizationName": "devopsabcs",
			"SourceProjectName": "WebJobs",
			"SourcePAT": "ZZZsourceOrgPATZZZ",
			"AreaPrefix": "Training\\CertAZ203",
			"TeamPrefix": "WJ_"
		}
	],
	"LoadEnginesPostCreation": true
}

A few comments are in order. Although the SourceOrganizationName for each project above is the same, namely, devopsabcs, this does not need to be the case. Indeed, the Migration Tool is capable of merging projects coming from multiple Azure DevOps organizations as long as you provide a valid Personal Access Token (PAT) for each source project. Discussion on the Migration Tool is deferred to Part 2 of this Blog series. Each AreaPrefix above consists of exactly two path parts following the pattern “{Portfolio Name}\\{Program Name}”. Though this may be a common desired configuration for the single host project, it does not need to be the case. We do allow for variance in the number of path parts. In addition, the AreaPrefix is used to specify the build and release folder structure to house the migrated pipelines as well as the query folder structure to contain the source project’s migrated queries. Finally, the TeamPrefix is used to prefix the migrated team names as well as the migrated repository names and service connection names.

The prefixes serve two purposes:

1. Avoid name clashes in the host or target project
2. More importantly, facilitate filtering of teams, repositories, and service connections

It’s instructive to view a before and after migration picture for a specific project. We choose the SmartHotel360 project which can be generated via the Azure DevOps Demo Generator. We used the Demo Generator to clone the SmartHotel360 project into our devopsabcs organization.

The following images paint the initial picture of SmartHotel360:

After using the Migration Tool, this is how the SmartHotel360 project got migrated into the single project OneProject07 of the single organization agileatscale.

The 3 teams got migrated and prefixed with “SH_“. The team members for each team got created (if they did not already exist in the target organization) and added to the corresponding team(s).

The SmartHotel360 iterations got migrated and placed under the node with path “OneProject07\\Demo\\Migrations\\SmartHotel360“. Observe that the AreaPrefix specified was “Demo\\Migrations“. In this case, the migration of iterations is non-trivial and highly dependent on the enterprise’s needs. Ideally, iterations should be shared across all enterprise teams. However, in practice, this may be a non-trivial task due to the many possible variations in the way each source project could have defined and configured its iteration node structure.

By contrast, the migration of the SmartHotel360 areas is more straightforward. We’ve simply migrated the source area structure under the path “OneProject07\\Demo\\Migrations\\SmartHotel360“.

Furthermore, we’ve fenced the “SH_SmartHotel 360 Team” to the path “OneProject07\\Demo\\Migrations\\SmartHotel360” thereby ensuring its members don’t have access to the other migrated projects. So unless a member had access to other projects prior to migration, that member will only be able to access work items under the above area path.

The “SH_SmartHotel 360 Team” iterations are prefixed with “OneProject07\\Demo\\Migrations” but have essentially been preserved.

Similarly, the “SH_SmartHotel 360 Team” areas are prefixed with “OneProject07\\Demo\\Migrations” but have essentially been preserved.

The service connections have been prefixed with “SH_” in order to avoid name clashes. It’s important to migrate service connections as they are often required dependencies for build and release definitions.

The repositories are also prefixed by “SH_“. Furthermore, we’ve set access control entries for the “SH_SmartHotel 360 Team”.

The migrated work items have area and iteration prefixed by “OneProject07\\Demo\\Migrations”.

The Shared Queries got migrated under the path “Shared Queries/Demo/Migrations/SmartHotel360”.

In addition, we’ve configured the following access control entries in order to limit the scope of the “SH_SmartHotel 360 Team”:

Finally, the build and release pipelines have been migrated under the expected folder structure and security permissions as can be seen here:

In conclusion, if you are thinking of moving to Azure DevOps, I encourage you to do things right from the start and consider the ideas put forth in this article and by the others mentioned. Do your due diligence! You won’t regret your decision as the platform is quite mature and still continually evolving.

If, like many, you have multiple projects in one or more organizations and are considering consolidating these projects into “One Project To Rule Them All”, then stay tuned for Part 2 of this Blog series where we will discuss the Migration Tool mentioned in this article.

Additional Exploration

• Plans (Agile at scale)
• Portfolio management
• Configure a hierarchy of teams
• Review team Delivery Plans