More Synching Requirements & an Open Source App Named PowerFolder

August 10, 2007 at 7:54 pm (Uncategorized)

I had some more time to think about what I am looking for in a directory synchronization tool. I had decided that the main requirements were:

  • Peer-to-Peer model: mostly for private synchronization (i.e. only for the same user) from multiple nodes, but also opening up opportunities for group sharing
  • Branch-per-node: each node will be a branch, but will have to have some sort of intelligent merging in order to deal with the potential problem of binary files growing out of control
  • HTTP support for firewall issues

Given those new requirements, I googled “BitTorrent Sync” and was surprised to find a project by the name of PowerFolder | Easy Secure File Transfer, Share, Sync and Online Storage Software I’m doubly thrilled to find that it is Open Source, although it appears to have a dual-licensing scheme which automatically makes me skeptical.

I’ll have to give the produce a peak and report back whether or not it meets my needs. Stay tuned.

Permalink Leave a Comment

Home Directory Sync

August 5, 2007 at 2:16 pm (digitalLife)

Ok, now I’m feeling a bit obsessed with this idea. It would make my life tremendously easier if I didn’t have to think about managing my different files. I just want them to always be available on all of my mahcines, but without having to change the way I work at all.

Last night, I touched on how to make Windows a little less ugly such that it will work well with the standard *nix directory layout. I also mentioned some resources that go in to detail about how they manage their CVS/SVN syncing issues. But then I started to realize those solutions have one fatal flaw – they still require you to “just cvs up at the end of the day.” I don’t even want to have to worry about that, because I won’t remember to do it.

Now I feel compelled to lay out my requirements for what would be the perfect home directory solution. Maybe I’ll call it “Distributed Home Dir” or something more creative, but along those lines:

  • Synchronization of most files should happen automatically, unless there is a conflict
  • Certain “large” files (usually binary) should not be automatically sync’ed
  • Somewhat intelligent defaults should exist to determine which files need to be synched up and which ones don’t
  • Options should exist for whether or not to have “full synching” ( constanly tracking changes a la Google Docs and keeping a record of the history), “just copy” (no history), “manual history” (manually choose to ‘commit’ changes), “ignore,” and probably others.
  • Bad practices should cause warnings along the lines of: “Saving history for these large binary files will take up lots of space, maybe you should just save the latest copy “
  • Files that are not automatically sync’ed should be able to be obtained on demand (assuming connection is available)
  • Most syncing occurs in the background at very frequent intervals (on par with gMail or Google Docs)
  • Each sync’ing node should be able to be a source as well
  • Each sync’ing source should be able to proxy another node as well
  • Multiple available nodes might afford a bump in bandwidth a la BitTorrent
  • “Installing” should be drop-dead simple, and maybe even pre-configured if you are able to access the install from one of your existing nodes
  • “Thin” nodes such as a flash drive or PDA should integrate well
  • Alerts might occur when you are starting to cause a conflict, at which time you can choose to branch (layman’s terms: “work on a second copy and merge later or keep a separate copy for this environment”)
  • Some “branches” will have to be intentional however, so as not to overwrite intentionally different types of files. For example, the %HOME%/Documents/My\ Google\ Gadgets folder you find in Windows.
  • “System tray” capabilities would be nice
  • Integration with other “file systems” (see the MacFuse Google Tech Talk for more info)
  • Solving the “Shared” problem would be nice (i.e. when we download pics from our family camera, they go in a shared home dir . . how do we get access to those resources on another machine)
  • Backed by established version control systems, most likely a distributed one like mercurial.
  • Encryption should be mandatory

That is probably more than enough to get things running. Hopefully I won’t have to write much code (otherwise this may never happen).

Permalink Leave a Comment

Fixing the ridiculous Windows “Home Directory” Directory Structure

August 4, 2007 at 11:03 pm (digitalLife)

I’ve really over-complicated my life at this point. I’ve turned my quaint little home into a freakin’ disorganized computer lab (in case my wife ever reads these posts, I should thank her for putting up with me). At the moment, I’m trying to get a handle on all the files all over the place.

I’ve gone down this road a million times before and never ended up tackling the problem sufficiently. However, I’m feeling inspired by a blog post I just found illustrating HOWTO rid yourself of the abhorrent “Documents and Settings” and “My Documents, My Music, My This and My That. ” I always thought my aversion to the Windows way of things was just because I was a command-line junkie, but I now realize their mess was part of the roadblock to organizing the various files on all these systems.

I’ve been hoping to use a source control solution like subversion (of course, after listening to Linus Torvalds at a Google Tech Talk, I’m thinking this is the perfect use case for a “distributed source control” solution). However, I’ve never wanted to go after bits and pieces at a time – I want the whole dang home directory synched – BOOM. I’ve never had the desire to play around with where to put various crap. Up until now, the best solution I could come up with was from one article I read which had a slick solution of checking in little set-up scripts for various environments. Still though, those scripts sound kind of ugly when Windows is all high maintenance.

Then comes the article: Derek Slager: Emulating Vista’s User Directory Structure on XP In a nutshell, instead of relying on the various scripts for the various environments to piece together your desired directory structure, just change the screwed up environment itself! If I can pull this off, then the change will also be able to be applied to the Home Directories of the wife and kids, which will be the proverbial grand slam of overly complicated home computing.

So, the plan is the following:

  1. Implement the changes from the referenced blog on all my XP instances
  2. Implement source control (some resources: http://toykeeper.net/tutorials/svnhome, http://tratt.net/laurie/tech_articles/articles/home_directory_synchronization, http://linuxjournal.com/article/5976)
  3. Make the conflict resolution process drop-dead simple for even my wife (who is actually pretty savy at computing, but hasn’t seen a diff in her life)

Honestly though, this will only solve some of my problems. I would still much rather prefer being able to just “mount” certain network directories to certain places (specifically in dealing with our shared pictures, as the size of them is too big to “distribute” to all machines).

Alas, let’s see where this adventure takes us.

Permalink Leave a Comment