One Project To Rule Them All

devopsabcsadmin

Part 3: Security

In Part 1 of the series (first showcased on Microsoft’s Premier Developer Blog), we discussed the business value of taking a single organization, single project approach in your Azure DevOps journey. In Part 2, we covered the fundamental techniques on which the Migration Tool relies. In this post, we will talk about security considerations and techniques.

Security Requirements can vary widely from one enterprise to another. Under certain circumstances, it may even be a reason to deviate from the “One Project To Rule Them All” strategy and add another project.

That being said, Azure DevOps offers a high level of granularity and control from the organization level all the way down to object-level permissions. For many enterprises, this is sufficient to adopt the “One Project To Rule Them All” approach.

Security is the 4th stage of the Migration Tool. In this stage, we take a simplified approach: give the migrated project’s default team the default Contributor permissions and access to all of the project’s migrated items. Though simplified, the approach is complex enough to illustrate the main techniques in dealing with security in Azure DevOps. An enterprise can easily adopt these techniques to implement different, more complex, security configurations. Furthermore, a tailored structured approach is an opportunity to consolidate and standardize your security strategy in Azure DevOps.

The heart of setting access control entries using the REST API is through the following HTTP POST call:

POST https://dev.azure.com/{organization}/_apis/accesscontrolentries/{securityNamespaceId}?api-version=5.0
Set Access Control Entries

As with many POST calls, setting access control entries requires a request body. The Microsoft Documentation provides this example:

{
  "token": "newToken",
  "merge": true,
  "accessControlEntries": [
    {
      "descriptor": "Microsoft.TeamFoundation.Identity;S-1-9-1551374245-1204400969-2402986413-2179408616-0-0-0-0-2",
      "allow": 8,
      "deny": 0,
      "extendedinfo": {}
    }
  ]
}

We will go over each detail, as each deserves special attention:

  • How does one obatin the {securityNamespaceId} of the POST call?
  • How to get the token in the request body?
  • How to get the descriptor in the request body? This question was asked in the Visual Studio Developer Community.
  • How to calculate the allow and deny integer values?

Security Namespaces

As the Microsoft Documentation states:

Security namespaces are used to store access control lists (ACLs) on tokens.

One can obtain a list of security namespaces with the following HTTP GET call:

https://dev.azure.com/{organization}/_apis/securitynamespaces?api-version=5.0

Most likely, you will obatin something similar to this JSON element for the Git Repositories Security Namespace:

        {
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87",
            "name": "Git Repositories",
            "displayName": "Git Repositories",
            "separatorValue": "/",
            "elementLength": -1,
            "writePermission": 8192,
            "readPermission": 2,
            "dataspaceCategory": "Git",
            "actions": [
                {
                    "bit": 1,
                    "name": "Administer",
                    "displayName": "Administer",
                    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
                },
                {
                    "bit": 2,
                    "name": "GenericRead",
                    "displayName": "Read",
                    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
                },
                {
                    "bit": 4,
                    "name": "GenericContribute",
                    "displayName": "Contribute",
                    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
                },
                {
                    "bit": 8,
                    "name": "ForcePush",
                    "displayName": "Force push (rewrite history, delete branches and tags)",
                    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
                },
                {
                    "bit": 16,
                    "name": "CreateBranch",
                    "displayName": "Create branch",
                    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
                },
                {
                    "bit": 32,
                    "name": "CreateTag",
                    "displayName": "Create tag",
                    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
                },
                {
                    "bit": 64,
                    "name": "ManageNote",
                    "displayName": "Manage notes",
                    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
                },
                {
                    "bit": 128,
                    "name": "PolicyExempt",
                    "displayName": "Bypass policies when pushing",
                    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
                },
                {
                    "bit": 256,
                    "name": "CreateRepository",
                    "displayName": "Create repository",
                    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
                },
                {
                    "bit": 512,
                    "name": "DeleteRepository",
                    "displayName": "Delete repository",
                    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
                },
                {
                    "bit": 1024,
                    "name": "RenameRepository",
                    "displayName": "Rename repository",
                    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
                },
                {
                    "bit": 2048,
                    "name": "EditPolicies",
                    "displayName": "Edit policies",
                    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
                },
                {
                    "bit": 4096,
                    "name": "RemoveOthersLocks",
                    "displayName": "Remove others' locks",
                    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
                },
                {
                    "bit": 8192,
                    "name": "ManagePermissions",
                    "displayName": "Manage permissions",
                    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
                },
                {
                    "bit": 16384,
                    "name": "PullRequestContribute",
                    "displayName": "Contribute to pull requests",
                    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
                },
                {
                    "bit": 32768,
                    "name": "PullRequestBypassPolicy",
                    "displayName": "Bypass policies when completing pull requests",
                    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
                }
            ],
            "structureValue": 1,
            "extensionType": "Microsoft.TeamFoundation.Git.Server.Plugins.GitSecurityNamespaceExtension",
            "isRemotable": true,
            "useTokenTranslator": true,
            "systemBitMask": 0
        }

Observe that there are 16 bits required to fully set all possible combinations of permissions on actions. Indeed, 15 of the 16 actions can be seen in the following (only the Administer action is missing):

Git Repositories securable actions

One small note, you may not see the ReleaseManagement Security Namespaces (no, there is no typo, you may see more than one!) in a newly created Azure DevOps organization. In order to see them, you may need to wake them up by actually creating a release definition or release.

In order to obtain the integer values for allow or deny, we simply use the enumeration types as bit flags. From the above response, we can easily generate the following enumeration:

    [System.Flags]
    public enum GitRepositoriesFlags
    {
        Administer = 1,
        GenericRead = 2,
        GenericContribute = 4,
        ForcePush = 8,
        CreateBranch = 16,
        CreateTag = 32,
        ManageNote = 64,
        PolicyExempt = 128,
        CreateRepository = 256,
        DeleteRepository = 512,
        RenameRepository = 1024,
        EditPolicies = 2048,
        RemoveOthersLocks = 4096,
        ManagePermissions = 8192,
        PullRequestContribute = 16384,
        PullRequestBypassPolicy = 32768,
    }

So if you’d like to secure an individual repository with Contributor permissions as in this screenshot:

Figure 1

In code, this translates to:

            int expected = 16502;
            var actualValue =
                GitRepositoriesFlags.GenericContribute |
                GitRepositoriesFlags.PullRequestContribute |
                GitRepositoriesFlags.CreateBranch |
                GitRepositoriesFlags.CreateTag |
                GitRepositoriesFlags.ManageNote |
                GitRepositoriesFlags.GenericRead;

            int actual = (int)actualValue;
            Assert.AreEqual(expected, actual);

The magic number for the allow value is 16502. If all we were looking for is the Contributor permissions, then we could have just as easily obtained the value by the following GET call:

https://dev.azure.com/agileatscale/_apis/accesscontrollists/2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87?api-version=5.0

Unfortunately, this gives me 1289 results, as I have this many access control lists in my agileatscale organization! I need to filter the above query with optional parameters:

GET https://dev.azure.com/{organization}/_apis/accesscontrollists/{securityNamespaceId}?token={token}&descriptors={descriptors}&api-version=5.0

In order to do so, we must answer two more questions we had set out to solve:

  • How to get the token in the request body?
  • How to get the descriptor in the request body?

Getting the token is highly dependent on the resource you are trying to secure. For Git Repository Tokens, I found this post useful by Matt Cooper. Note that we have the repositoryId 0f22acb2-4c10-4e79-84d3-69dd0d798412 from the URL of Figure 1. The following GET call:

https://dev.azure.com/agileatscale/oneproject07/_apis/git/repositories/0f22acb2-4c10-4e79-84d3-69dd0d798412?api-version=5.0

yields this JSON response:

{
    "id": "0f22acb2-4c10-4e79-84d3-69dd0d798412",
    "name": "AS_PS.MAD.D4AS",
    "url": "https://dev.azure.com/agileatscale/fe374bc1-e0ad-4ed9-a35e-d8d1564e554e/_apis/git/repositories/0f22acb2-4c10-4e79-84d3-69dd0d798412",
    "project": {
        "id": "fe374bc1-e0ad-4ed9-a35e-d8d1564e554e",
        "name": "OneProject07",
        "description": "full regression cmd line",
        "url": "https://dev.azure.com/agileatscale/_apis/projects/fe374bc1-e0ad-4ed9-a35e-d8d1564e554e",
        "state": "wellFormed",
        "revision": 2493,
        "visibility": "private",
        "lastUpdateTime": "2019-06-09T22:44:22.407Z"
    },
    "defaultBranch": "refs/heads/master",
    "size": 101250,
    "remoteUrl": "https://agileatscale@dev.azure.com/agileatscale/OneProject07/_git/AS_PS.MAD.D4AS",
    "sshUrl": "git@ssh.dev.azure.com:v3/agileatscale/OneProject07/AS_PS.MAD.D4AS",
    "webUrl": "https://dev.azure.com/agileatscale/OneProject07/_git/AS_PS.MAD.D4AS",
    "_links": {
        "self": {
            "href": "https://dev.azure.com/agileatscale/fe374bc1-e0ad-4ed9-a35e-d8d1564e554e/_apis/git/repositories/0f22acb2-4c10-4e79-84d3-69dd0d798412"
        },
        "project": {
            "href": "vstfs:///Classification/TeamProject/fe374bc1-e0ad-4ed9-a35e-d8d1564e554e"
        },
        "web": {
            "href": "https://dev.azure.com/agileatscale/OneProject07/_git/AS_PS.MAD.D4AS"
        },
        "ssh": {
            "href": "git@ssh.dev.azure.com:v3/agileatscale/OneProject07/AS_PS.MAD.D4AS"
        },
        "commits": {
            "href": "https://dev.azure.com/agileatscale/fe374bc1-e0ad-4ed9-a35e-d8d1564e554e/_apis/git/repositories/0f22acb2-4c10-4e79-84d3-69dd0d798412/commits"
        },
        "refs": {
            "href": "https://dev.azure.com/agileatscale/fe374bc1-e0ad-4ed9-a35e-d8d1564e554e/_apis/git/repositories/0f22acb2-4c10-4e79-84d3-69dd0d798412/refs"
        },
        "pullRequests": {
            "href": "https://dev.azure.com/agileatscale/fe374bc1-e0ad-4ed9-a35e-d8d1564e554e/_apis/git/repositories/0f22acb2-4c10-4e79-84d3-69dd0d798412/pullRequests"
        },
        "items": {
            "href": "https://dev.azure.com/agileatscale/fe374bc1-e0ad-4ed9-a35e-d8d1564e554e/_apis/git/repositories/0f22acb2-4c10-4e79-84d3-69dd0d798412/items"
        },
        "pushes": {
            "href": "https://dev.azure.com/agileatscale/fe374bc1-e0ad-4ed9-a35e-d8d1564e554e/_apis/git/repositories/0f22acb2-4c10-4e79-84d3-69dd0d798412/pushes"
        }
    }
}

So the token we are looking for is:

repoV2/fe374bc1-e0ad-4ed9-a35e-d8d1564e554e/0f22acb2-4c10-4e79-84d3-69dd0d798412

So if we run the following query with the above token:

https://dev.azure.com/agileatscale/_apis/accesscontrollists/2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87?token=repoV2/fe374bc1-e0ad-4ed9-a35e-d8d1564e554e/0f22acb2-4c10-4e79-84d3-69dd0d798412&api-version=5.0

we get the JSON response:

{
    "count": 1,
    "value": [
        {
            "inheritPermissions": true,
            "token": "repoV2/fe374bc1-e0ad-4ed9-a35e-d8d1564e554e/0f22acb2-4c10-4e79-84d3-69dd0d798412",
            "acesDictionary": {
                "Microsoft.IdentityModel.Claims.ClaimsIdentity;a34c69c7-8959-474a-9690-e98bfb0b55c6\\emmanuel@devopsabcs.com": {
                    "descriptor": "Microsoft.IdentityModel.Claims.ClaimsIdentity;a34c69c7-8959-474a-9690-e98bfb0b55c6\\emmanuel@devopsabcs.com",
                    "allow": 32382,
                    "deny": 0
                },
                "Microsoft.TeamFoundation.Identity;S-1-9-1551374245-3242932222-2917194062-2740902097-1447974222-1-3578883301-4072410959-2197538308-2551652260": {
                    "descriptor": "Microsoft.TeamFoundation.Identity;S-1-9-1551374245-3242932222-2917194062-2740902097-1447974222-1-3578883301-4072410959-2197538308-2551652260",
                    "allow": 16502,
                    "deny": 0
                }
            }
        }
    ]
}

We do see the value 16502 but how can we be sure of the SID? To get the descriptor of the group we are trying to attribute permissions to, we must obtain its descriptor.

We do this by fetching all groups using the following GET call:

GET https://vssps.dev.azure.com/{organization}/_apis/graph/groups?api-version=5.0-preview.1

If you have many groups, you may only get 500, in which case, you will need to use a continuation token to fetch the next page of results. Look for the continuation token in the response headers:

Continuation token while fetching all groups

The continuation token I got was:

eyJTY29wZUlkIjoiNmQxMDQ3NTEtNWZhNi00ZDY5LTlhMTEtNWFiOWMxYjU3OTM2IiwiUGFnZVNpemUiOjUwMCwiSW5jbHVkZUdyb3VwcyI6dHJ1ZSwiSW5jbHVkZU5vbkdyb3VwcyI6ZmFsc2UsIlBhZ2VuYXRpb25Ub2tlbiI6ImNlZWYwZWRiLWJmNzQtNDc3Yi05NzA4LTdlM2MyNDY2ZDQ1OCJ9

which when decoded from base64 yields:

{
	"ScopeId": "6d104751-5fa6-4d69-9a11-5ab9c1b57936",
	"PageSize": 500,
	"IncludeGroups": true,
	"IncludeNonGroups": false,
	"PagenationToken": "ceef0edb-bf74-477b-9708-7e3c2466d458"
}

Observe that the PagenationToken is the originId of the last group fetched in the first 500 results.

However, this discussion is about Security, so if you are paranoid like me and think that this generous website may be recording your sensitive strings, then here are two quick methods that will allow you to achieve the same from the comfort of your trusted PC:

public static string Base64Decode(string base64EncodedData)
        {
            var lengthMod4 = base64EncodedData.Length % 4;
            if (lengthMod4 != 0)
            {
                //fix Invalid length for a Base-64 char array or string
                base64EncodedData += new string('=', 4 - lengthMod4);
            }
            var base64EncodedBytes = System.Convert.FromBase64String(base64EncodedData);
            return System.Text.Encoding.UTF8.GetString(base64EncodedBytes);
        }

public static string Base64Encode(string plainText)
        {
            var plainTextBytes = System.Text.Encoding.UTF8.GetBytes(plainText);
            return System.Convert.ToBase64String(plainTextBytes);
        }

I did find the group in question, namely, [OneProject07]\AS_Autoscaling Team in the second page of results:

        {
            "subjectKind": "group",
            "description": "The default project team.",
            "domain": "vstfs:///Classification/TeamProject/fe374bc1-e0ad-4ed9-a35e-d8d1564e554e",
            "principalName": "[OneProject07]\\AS_Autoscaling Team",
            "mailAddress": null,
            "origin": "vsts",
            "originId": "06f50375-1236-43ef-8294-be7f3d739e6c",
            "displayName": "AS_Autoscaling Team",
            "_links": {
                "self": {
                    "href": "https://vssps.dev.azure.com/agileatscale/_apis/Graph/Groups/vssgp.Uy0xLTktMTU1MTM3NDI0NS0zMjQyOTMyMjIyLTI5MTcxOTQwNjItMjc0MDkwMjA5Ny0xNDQ3OTc0MjIyLTEtMzU3ODg4MzMwMS00MDcyNDEwOTU5LTIxOTc1MzgzMDgtMjU1MTY1MjI2MA"
                },
                "memberships": {
                    "href": "https://vssps.dev.azure.com/agileatscale/_apis/Graph/Memberships/vssgp.Uy0xLTktMTU1MTM3NDI0NS0zMjQyOTMyMjIyLTI5MTcxOTQwNjItMjc0MDkwMjA5Ny0xNDQ3OTc0MjIyLTEtMzU3ODg4MzMwMS00MDcyNDEwOTU5LTIxOTc1MzgzMDgtMjU1MTY1MjI2MA"
                },
                "membershipState": {
                    "href": "https://vssps.dev.azure.com/agileatscale/_apis/Graph/MembershipStates/vssgp.Uy0xLTktMTU1MTM3NDI0NS0zMjQyOTMyMjIyLTI5MTcxOTQwNjItMjc0MDkwMjA5Ny0xNDQ3OTc0MjIyLTEtMzU3ODg4MzMwMS00MDcyNDEwOTU5LTIxOTc1MzgzMDgtMjU1MTY1MjI2MA"
                },
                "storageKey": {
                    "href": "https://vssps.dev.azure.com/agileatscale/_apis/Graph/StorageKeys/vssgp.Uy0xLTktMTU1MTM3NDI0NS0zMjQyOTMyMjIyLTI5MTcxOTQwNjItMjc0MDkwMjA5Ny0xNDQ3OTc0MjIyLTEtMzU3ODg4MzMwMS00MDcyNDEwOTU5LTIxOTc1MzgzMDgtMjU1MTY1MjI2MA"
                }
            },
            "url": "https://vssps.dev.azure.com/agileatscale/_apis/Graph/Groups/vssgp.Uy0xLTktMTU1MTM3NDI0NS0zMjQyOTMyMjIyLTI5MTcxOTQwNjItMjc0MDkwMjA5Ny0xNDQ3OTc0MjIyLTEtMzU3ODg4MzMwMS00MDcyNDEwOTU5LTIxOTc1MzgzMDgtMjU1MTY1MjI2MA",
            "descriptor": "vssgp.Uy0xLTktMTU1MTM3NDI0NS0zMjQyOTMyMjIyLTI5MTcxOTQwNjItMjc0MDkwMjA5Ny0xNDQ3OTc0MjIyLTEtMzU3ODg4MzMwMS00MDcyNDEwOTU5LTIxOTc1MzgzMDgtMjU1MTY1MjI2MA"
        }

Now if we decode the descriptor using our trusted C# code:

Uy0xLTktMTU1MTM3NDI0NS0zMjQyOTMyMjIyLTI5MTcxOTQwNjItMjc0MDkwMjA5Ny0xNDQ3OTc0MjIyLTEtMzU3ODg4MzMwMS00MDcyNDEwOTU5LTIxOTc1MzgzMDgtMjU1MTY1MjI2MA

we get the SID:

S-1-9-1551374245-3242932222-2917194062-2740902097-1447974222-1-3578883301-4072410959-2197538308-2551652260

This is great! It confirms that we had the right SID in the first place.

Finally, we’d like to mention one more thing. While we have tested the migration tool with Azure DevOps Services migration, with a little bit more effort, the tool can be made to work on-premise e.g. with Azure DevOps Server 2019. This is because the on-premise version also supports the REST API calls. The only caveat is that we must use the TFSSecurity.exe tool instead of the REST API for security related functionality. That being said, many of the lessons learned here can be carried over with the TFSSecurity.exe tool such as obtaining the security tokens.

Additional Exploration

1 thought on “One Project To Rule Them All”

Leave a Reply

Your email address will not be published. Required fields are marked *