Real-world use of Mercurial with a Team Foundation Server?

My shop uses TFS & is generally happy with it with the exception of the lack of local repository commits/reverts. I'm starting to use Mercurial locally myself to help manage smaller chunks of changes, then posting them to TFS. I see that Subversion has a 'bridge' component to automagically enable this if the central VCS is Subversion. I have not found one for Team System. This encourages me that other folks have gone down this path with integrating DVCS with CVCS systems.

(1) Does anyone know of one? I'm sort of doubting it (quick search didn't find anything).

(2) Is anyone using Mercurial/TFS in this manner? If so, can you share your experiences. I'm particularly looking for any insights into what issues might come up that aren't obvious regarding commits to TFS after significant activity via Mercurial.

It seems like a total win-win so far with just me using if for a few days--but I know enough then to think it's just that easy.

Not sure if this is anything you don't already know, but I've been using mercurial locally for a little while now, and so far I think the benefits are outweighing the added overhead of managing the two source control systems. Here is how I've been doing things:

1. I made my TFS checkout an HG repository which I consider my "master". I get updates from TFS and commit them to this repo, so this contains the most current state of the project from TFS. The important thing here is that there are no changes to this made independent of a TFS update or an Hg merge (which is part 2)

2. Anytime I need to make a change, I clone my "master" repo and do my work there. I've found that a clone per feature or story is actually pretty easy to manage, and feels pretty clean. Once I complete a feature, I do an Hg merge back to the "master" repo, which has had all of the TFS updates applied. This allows me to use Mercurials merge capabilities, which are so far superior to TFS as to call into question how TFS can claim to merge code at all. Once the merge is complete, I commit it in Hg, and then check those changes into TFS. The best part of this is that when I do the checkin to TFS, I do not have to merge anything. Very, very nice.

Now, here are the issues I've found with this approach:

1. The biggest is the fact that TFS is lousy at finding changes. There is a make writable plugin which you can use to make the modified files writable when they are updated/merged by Mercurial. There are two options I have found for this. You can either force TFS to go offline, at which point it will assume anything writable needs to be checked in, or you can use the compare tool in the source control tool and select the changed files and individually check them out. Both are crappy IMO

2. The source control bindings are still there at the project level, even if you exclude the TFS source control files from your hg repository (which you should do). This isn't entirly obvious until you add a file to the solution, at which point it tries adding it to source control. You can "Undo Pending Changes" and get rid of the source control add, but it's really annoying.

The good news is that I used this approach to work through a rather massive merge which I think would have caused me to turn to some form of hard drugs had I been forced to use the TFS tools to do it.

I have not yet applied this to updating branches within TFS, but my guess would be that it would be much much better than the options you are given for merging in TFS. On a related note, since you can check in chunks of working functionality at one time, using the TFS merge would be less problematic just because all changes needed for a feature would be together in one place.

One thing I have not tried to tackle is sharing this across the entire team. Part of the reason is that it really doesn't have to be a team-wide thing. I work remotely, so having a local repository is a big deal, and saves a lot of time. The other members of my dev team may or may not get the same benefit from this approach, but I find it pretty cool that I can without effecting the way they work.

Update I've been wanting to update this response for a while with additional info based on comments and some of my experiences working with large TFS repositories.

First as @Eric Hexter points out in the comments, you can utilize the rebase extension to better integrate commits from your work repositories into your main TFS repository. Though, depending on how you want your commits to appear to TFS you may want to use the collapse extension to squash your changes into a single commit (this can make rollbacks in TFS easier). There is also the "online" command from TFS PowerTools which can make the work of letting TFS know what has changed easier (thanks again to Eric for mentioning that in his blog post)

Now, when I originally wrote this I was working on a project that had only one TFS branch that developers were using, and was fairly small, so cloning repositories was no big deal. I later found myself working on a project that had a repo that was about 1.5GB after checkout, and much bigger after the build, and involved switching between branches in TFS quite often. Clearly this approach is not well suited to this environment (particularly since at one point it was impossible to build the solutions in an arbitrary directory.

The size problem is best handled by utilizing a technique similar to gits topic branches rather than cloning the repositories to new directories. There are a couple options for this. I think the best is actually to use the bookmark extension and create topic "bookmarks" rather than topic branches. You can use named branches as well, but they have the slight disadvantage of being permanent, and traveling with any clones you may do (if you want to share your nifty TFS-Hg hybrid with a colleague). Bookmarks are local to your repo, and effectively point to a commit and travel with the head. They are implemented so that they can be used in any place where Hg is expecting a revision (so merges, updates, etc). You can use these to create a TFS bookmark as your primary "branch" that only gets updates from TFS, and merges from the topic work, which would each have their own bookmarks, and you could delete once you have committed back to TFS. If you would rather use named branches, then you can apply exactly the same techniques, which is handy.

Now, the multiple branch problem is trickier, especially since TFS "branches" are actually copies of every file from the original branch, which means that each time you pull in branches from TFS, your repo is going to get that much bigger. One possible way to deal with this is use a combination of named Hg branches, and bookmarks, so that you have a branch for each TFS branch, and then create bookmarks for your work off of those branches. The real headache in these scenarios is actually dealing with TFS workspaces through all of this. You can remove the mappings in your workspaces and get pretty far, but once you map back to your working directory you've got to be careful of TFS stomping on files (this is actually where the TF PowerTools come in handy). Trying to leave the workspace attached while your switching branches around gets ugly quick. A couple tools that are nice to have in your toolbelt are the Hg purge extension and the TF PowerTools "scorch" command. Both effectivly remove files that are not in version control (technically "scorch" ensure TFS and your local working directory match, so it could update files as well).

For me, though, this process became quite burdensome and error prone. I've recently switched to useing git with git-tfs, since it manages TFS Workspaces for me, and removes a lot of the burden associated with that side. Sadly there does not appear to be an "hg-tfs" out there anywhere, or I would probably have chosen that.

If you're not stuck on mercurial, there is a sweet git/tfs integration project that I've been using called git-tfs. It's very similar to git-svn, but pushes/pulls from TFS instead. Check it out at http://github.com/spraints/git-tfs

@Eric, your post at lostechies was most helpful. With VS2010 I had to add options /diff and /deletes to the tftp online command in the push script to get changed and deleted files to be checked in to TFS. Initially I was getting an error from push when a file has been deleted (from -working) that hg update is
"unable to remove FileXyz : access is denied".
I installed the MakeWritable.py extension but that only works when files are opened not deleted. So I added a call to attrib to remove the READ-ONLY from all files in the project and then restore it afterwards (excluding the .hg folder) I also added the /diff option so that differences are detected by MD5 checksum instead of depending on the READ-ONLY attribute. Seems to be working fine now.

=====FILE: push.ps1=====
$projName = "TicTacToeCMMI"
$tftp = "C:\Program Files\Microsoft Team Foundation Server 2010 Power Tools\TFPT.exe"
$tf = "C:\Program Files\Microsoft Visual Studio 10.0\Common7\ide\tf.exe"

hg push
cd ..\$projName-tfs  
"Syncing -tfs workspace with TFS server"  
&$tftp scorch /noprompt /exclude:.hg',_Resharper*',*.user  
"Making all files in -tfs writable"
attrib -R /S /D *
"Updating -tfs with latest push from Mercurial"
hg update -C -y
attrib +R /S /D *
attrib -R /S /D .hg\*
"Resyncing Mercurial changes with TFS Server"  
&$tftp online /adds /deletes /diff /exclude:'.hgignore,.hg,bin,obj,*.ps1,_Resharper*,*.lnk,*.user,*.suo,*.vspscc'  
"Checkin"  
&$tf checkin  
cd ..\$projName-working  
cmd /c pause  

====FILE: pull.ps1=====
$projName = "TicTacToeCMMI"
$tf = "C:\Program Files\Microsoft Visual Studio 10.0\Common7\ide\tf.exe"
$username = cmd /c set USERNAME
$username = $username.SubString($username.IndexOf("=")+1)

function pull {
    cd ..\$projName-tfs
    &$tf get
    hg commit -A -m "from tfs" --user $username
    cd ..\$projName-working
    hg pull --rebase
}
pull  
cmd /c pause  

I had a bit of a learning curve with PowerShell scripts which I hadn't used before. For others like me the scripts are run with a shortcut like this:

TARGET: C:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe C:\dev\TicTacToeCMMI-working\push.ps1
START IN: C:\dev\TicTacToeCMMI-working

I put push and pull shortcuts on my task bar so push/pull to/from TFS is a single click

I know that some people have used hgsubversion with the Subversion bridge. I don't know how well it worked, and I've never had to use TFS.

As far as I'm aware, there's no "more native" bridge than using TFS -> Subversion Bridge -> hgsubversion, but I've also heard that it works fairly well. My extremely limited understanding of TFS suggests that its internal model should be similar enough to Subversion for things like hgsubversion to work really well.

If you want to be able to work with a DVCS and TFS, I believe the best way is to install the SVNBridge for TFS, and the use Bazaar, which is, AFAIK the only DVCS that easily integrates with SVN, and since your TFS now looks like a SVN, you magically get Bazaar/TFS integration

Here's a powershell script I've been using to work with TFS & hg. To use you'll need to create a hg repository in your TFS folder (commit the files from TFS into it), clone this repository and work on the new repository. Once happy you can run "hgtfs.ps1 push" to push the changes back into TFS from your mercurial repository.

hgtfs.ps1:

param([parameter(Position=0, Mandatory=$true)][string] $action)

$HGDirectory = Get-Location
$TfsDirectory = @(hg paths | where-object { $_.StartsWith("default = ") })[0].SubString(10)

# Pull from TFS
function pull
{
    # Todo pull changes one by one brining who did it and the comment into HG
    # tf history . /recursive /format:brief /noprompt /version:300~1000 /sort:ascending
    # tf properties . /recursive

    # Add the changes from TFS into the TFS HG repository
    Set-Location $TfsDirectory
    tf get . /recursive
    hg commit -A -m "Update from TFS"  
    
    # Pull / merge the changes from TFS's HG repository
    Set-Location $HGDirectory
    hg pull
    hg merge --tool internal:fail
    hg commit -m "Merged from TFS"
    
    ""
    "The you have the following conflicts which need resolving"
    hg resolve -l | write-host -foregroundcolor "red"
    #thg commit
}

# Push to TFS
function push 
{
    Set-Location $HGDirectory
    hg push
    Set-Location $TfsDirectory

    $FilesModified = @()
    $FilesRenamed = @{} # Key: old file name .... Val: new file name
    $FilesRemoved = @()
    $FilesAdded = @()

    # Work out what changes have taken place
    "Calculating the changes which have been made in HG..."
    tfpt scorch /exclude:.hg,*.user | out-null
    $AllChanges = hg status --rev .:tip -A 
    for($i = 0; $i -lt $AllChanges.length ; $i++)
    {
        $type = $AllChanges[$i].SubString(0, 2)
        $fileName = $AllChanges[$i].SubString(2)
            
        switch($type)
        {
            "M " # Modified files  
                { 
                    $FilesModified += $fileName
                } 

            "A " # New Files
                {  
                    $nextType = $null
                    $nextFileName = $null
                    if($AllChanges.length -gt ($i+1))
                    {
                        $nextType = $AllChanges[$i+1].SubString(0, 2)
                        $nextFileName = $AllChanges[$i+1].SubString(2)                
                    }
                    
                    if($nextType -eq "  ")
                    {
                        # we have a rename
                        $FilesRenamed[$nextFileName]=$fileName
                        $i++
                    }
                    else
                    {
                        # we're adding the file
                        $FilesAdded += $fileName
                    }
                 }
                 
            "R " # Removed
                {
                    if($FilesRenamed.ContainsKey($fileName))
                    {
                        continue
                    }
                    
                    $FilesRemoved += $fileName
                }
            
            "C " # Same 
                { 
                    continue 
                }
                 
            default 
                { 
                    "Unknown HG status line: "+$AllChanges[$i] 
                    return -1
                }
        }
    }

    # perform the TFS operations 
    "Renaming files in TFS..."
    foreach($file in $FilesRenamed.Keys) {   
        tf checkout $file | out-null
        tf rename $file $FilesRenamed[$file] | out-null
    }
    
    "Checking out for edit in TFS..."
    foreach($file in $FilesModified) { tf checkout $file | out-null }
    
    "Removing files from TFS..."
    foreach($file in $FilesRemoved) { tf delete $file | out-null }

    # perform the Mercural update
    "Pulling changes out of HG...."
    hg update --rev .:tip --clean

    # perform any POST TFS operations
    "Adding new files to TFS..."
    foreach($file in $FilesAdded) { tf add $file }
    
    "Cleaning up..."
    tfpt uu /noget
    tf checkin
}


if ($action -eq "push") { push }
elseif ($action -eq "pull") { pull }
else { "Unknown action ... please supply 'push' or 'pull'" }

# return to our starting point
Set-Location $HGDirectory

I'he just put together a small tool, HgTfs, which tries to accomplish the goal of syncing Mercurial and TFS repositories. It's really simple and and has only three commands: clone, pull and push. Here is my Bitbucket repo:

https://bitbucket.org/thepretender/hgtfs

There is also a blog post describing the workflow and usage scenarios (actually, wiki pages are just parts of this blog entry):

http://www.olegtarasov.me/Post/2013/07/Mercurial-to-TFS-bridge-(hgtfs)

The code is hacky, but it seems to get the job done. I would really appreciate any feedback or forks :)

I had a good try at getting it working. I could get Git and TFS playing together (link) via svnbridge, but I couldn't get mercurial to work via svnbridge which frustrated me no end. If you manage to get it working let me know, because I personally prefer mercurial over git (though both are great)