OneNote 2007 With WebDAV over HTTPS
OneNote 2007 is out, and of course I rushed out to get a copy. It was the main reason that I had participated in the Office 2007 beta, and there were a number of great new feature. One particularly great new feature is the ability to have notes stored on a server, with clients syncing up against it. For me, this means that I can use my notes on my laptop at home, or on my desktop at work, without having to manually sync thing. Chris Pratley, the ex-manager of the OneNote team, wrote a very good description of all of the new multi-machine possibilities. Good good stuff: Any WebDAV server can acts as the repository. But..
However, I'm wary of using a WebDAV server across the internet. I might have very private things in my notes (e.g. account numbers, etc), and sending them around unencrypted doesn't sound like a good idea. Dan Escapa of Microsoft pointed out that Vista doesn't even allow SharePoint access through the internet via unsecured HTTP. David Rasmussen, also of the OneNote team, wrote about using flash USB drives as a way of keeping things in sync, an alternative to the network. (In the comments of that post, he mentions that password-protecting the .one files might keep things private.)
Unfortunately, when I tried to use WebDAV over HTTPS (with Basic Authentication) during the Beta, it failed. Looking through my webserver logs (I was running Apache on Debian), OneNote seemed to occasionally make requests over to the unsecured HTTP URLs. I discussed this with Dave and another MS developer, and they did some testing.
So when I tried it out with the production version of OneNote, I was disappointed to find that the error still persists. Let me try to detail what the problem is. The MS team may have made some changes since the Beta, as I think that I have a way to work around it.
I have complete control over my server (Apache 2.0.54), and am running OneNote 2007 (build 12.0.4518.1014). I've split my website into an area that has my normal content (www.borlik.net), and an area that just has WebDAV content (webdav.borlik.net) using virtual hosts. In addition, each of those areas has separate virtual hosts for HTTP and HTTPS. Originally, I didn’t want to have an unsecure WebDAV location at all, and hoped to simply redirect all HTTP traffic to HTTPS.
I tested this by trying to create a new OneNote notebook (named xxx) at the secure webdav location, e.g. https://webdav.borlik.net/. This failed with message boxes. If you look carefully at the webserver logs, one might notice something odd in what OneNote is requesting from the server:
192.168.0.148 - jborlik [01/Feb/2007:20:58:13 -0800] "HEAD / HTTP/1.1" 200 - "-" "Microsoft Office Existence Discovery" 192.168.0.148 - jborlik [01/Feb/2007:20:58:13 -0800] "HEAD /xxx/ HTTP/1.1" 404 - "-" "Microsoft Office Existence Discovery" 192.168.0.148 - jborlik [01/Feb/2007:20:58:13 -0800] "HEAD / HTTP/1.1" 200 - "-" "Microsoft Office Existence Discovery" 192.168.0.148 - jborlik [01/Feb/2007:20:58:14 -0800] "OPTIONS / HTTP/1.1" 200 - "-" "Microsoft Data Access Internet Publishing Provider Protocol Discovery" 192.168.0.148 - jborlik [01/Feb/2007:20:58:14 -0800] "OPTIONS /xxx HTTP/1.1" 200 - "-" "Microsoft Data Access Internet Publishing Provider Protocol Discovery" 192.168.0.148 - jborlik [01/Feb/2007:20:58:14 -0800] "HEAD /xxx HTTP/1.1" 404 - "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:20:58:14 -0800] "MKCOL /xxx HTTP/1.1" 201 324 "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:20:58:14 -0800] "HEAD /xxx/ HTTP/1.1" 200 - "-" "Microsoft Office Existence Discovery" 192.168.0.148 - jborlik [01/Feb/2007:20:58:14 -0800] "HEAD /xxx/ HTTP/1.1" 200 - "-" "Microsoft Office Existence Discovery" 192.168.0.148 - jborlik [01/Feb/2007:20:58:14 -0800] "HEAD /xxx/ HTTP/1.1" 200 - "-" "Microsoft Office Existence Discovery" 192.168.0.148 - jborlik [01/Feb/2007:20:58:14 -0800] "OPTIONS /xxx/ HTTP/1.1" 200 - "-" "Microsoft Office Protocol Discovery" 192.168.0.148 - jborlik [01/Feb/2007:20:58:14 -0800] "PROPFIND /xxx/New%20Section%201.one HTTP/1.1" 404 363 "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:20:58:14 -0800] "OPTIONS / HTTP/1.1" 200 - "-" "Microsoft Data Access Internet Publishing Provider Protocol Discovery" 192.168.0.148 - jborlik [01/Feb/2007:20:58:14 -0800] "OPTIONS /xxx/New%20Section%201.one HTTP/1.1" 200 - "-" "Microsoft Data Access Internet Publishing Provider Protocol Discovery" 192.168.0.148 - - [01/Feb/2007:20:58:14 -0800] "OPTIONS / HTTP/1.1" 301 377 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600" 192.168.0.148 - jborlik [01/Feb/2007:20:58:14 -0800] "PROPFIND /xxx HTTP/1.1" 207 837 "-" "Microsoft Data Access Internet Publishing Provider DAV"
Can you find it? Most of the requests are on the secured (HTTPS) URL, authenticated with my username (jborlik). However, the second to last request is an OPTION request to the root of the HTTP site. Apache returns a 301 (redirect permanent), as it should, and OneNote stops. I don't know what it is looking for at the root, and I really don't want there to be anything there. On the server, a directory for the new notebook is actually created, but none of the template .one files are created. OneNote itself displays a red crossed-out circle over the notebook, indicating that it can't sync.
If I do not redirect HTTP to HTTPS, i.e. open the WebDAV directory to unauthenticated access, things seem to work fine. (I'm doing the same thing: Creating a new notebook "yyy" at the secured URL.)
… 192.168.0.148 - jborlik [01/Feb/2007:21:02:42 -0800] "OPTIONS /yyy/New%20Section%201.one HTTP/1.1" 200 - "-" "Microsoft Data Access Internet Publishing Provider Protocol Discovery" 192.168.0.148 - - [01/Feb/2007:21:02:42 -0800] "OPTIONS / HTTP/1.1" 200 - "-" "Microsoft-WebDAV-MiniRedir/5.1.2600" 192.168.0.148 - - [01/Feb/2007:21:02:42 -0800] "PROPFIND /yyy HTTP/1.1" 207 837 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600" 192.168.0.148 - - [01/Feb/2007:21:02:42 -0800] "PROPFIND /yyy HTTP/1.1" 207 837 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600" 192.168.0.148 - jborlik [01/Feb/2007:21:02:42 -0800] "PROPFIND /yyy HTTP/1.1" 207 837 "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - - [01/Feb/2007:21:02:42 -0800] "PROPFIND /yyy HTTP/1.1" 207 837 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600" 192.168.0.148 - jborlik [01/Feb/2007:21:02:42 -0800] "HEAD /yyy/OneNote%20Table%20Of%20Contents.onetoc2 HTTP/1.1" 404 - "-" "Microsoft Office Existence Discovery" 192.168.0.148 - jborlik [01/Feb/2007:21:02:42 -0800] "HEAD /yyy HTTP/1.1" 301 - "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:21:02:42 -0800] "MKCOL /yyy HTTP/1.1" 405 370 "-" "Microsoft Data Access Internet Publishing Provider DAV" ...
At the critical times, OneNote makes the OPTIONS and PROPFIND requests over unauthenticated HTTP. Those are successful now (code 200), so it moves on does the rest of its thing. Of course, this isn't secure at all, and we are back to where we were before.
The next thing that I tried was to only allow OPTIONS and PROPFIND requests over the unauthenticated HTTP. These aren't terribly unsecure, I suppose, especially if GET and PUT are rejected. This is done in Apache via the LimitExcept directives. So, my unsecured WebDAV virtual host definition contains something that looks like:
<Location />
Options Indexes
Dav on
DavDepthInfinity on
AllowOverride None
<LimitExcept OPTIONS PROPFIND>
deny from all
</LimitExcept>
</Location>
When I tried the same thing again (created a new notebook "ccc" on the secure URL), it actually seemed to work!!
192.168.0.148 - jborlik [01/Feb/2007:21:28:56 -0800] "HEAD / HTTP/1.1" 200 - "-" "Microsoft Office Existence Discovery" 192.168.0.148 - jborlik [01/Feb/2007:21:28:56 -0800] "HEAD /ccc/ HTTP/1.1" 404 - "-" "Microsoft Office Existence Discovery" 192.168.0.148 - jborlik [01/Feb/2007:21:28:56 -0800] "HEAD / HTTP/1.1" 200 - "-" "Microsoft Office Existence Discovery" 192.168.0.148 - jborlik [01/Feb/2007:21:28:56 -0800] "OPTIONS / HTTP/1.1" 200 - "-" "Microsoft Data Access Internet Publishing Provider Protocol Discovery" 192.168.0.148 - jborlik [01/Feb/2007:21:28:56 -0800] "OPTIONS /ccc HTTP/1.1" 200 - "-" "Microsoft Data Access Internet Publishing Provider Protocol Discovery" 192.168.0.148 - jborlik [01/Feb/2007:21:28:56 -0800] "HEAD /ccc HTTP/1.1" 404 - "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:21:28:56 -0800] "MKCOL /ccc HTTP/1.1" 201 324 "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:21:28:56 -0800] "HEAD /ccc/ HTTP/1.1" 200 - "-" "Microsoft Office Existence Discovery" 192.168.0.148 - jborlik [01/Feb/2007:21:28:56 -0800] "HEAD /ccc/ HTTP/1.1" 200 - "-" "Microsoft Office Existence Discovery" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "HEAD /ccc HTTP/1.1" 301 - "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "MKCOL /ccc HTTP/1.1" 405 370 "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "PROPFIND / HTTP/1.1" 207 6474 "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "HEAD /ccc HTTP/1.1" 301 - "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "MKCOL /ccc HTTP/1.1" 405 370 "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "HEAD /ccc/ HTTP/1.1" 200 - "-" "Microsoft Office Existence Discovery" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "HEAD /ccc/Client%20A.one HTTP/1.1" 404 - "-" "Microsoft Office Existence Discovery" 192.168.0.148 - - [01/Feb/2007:21:28:57 -0800] "OPTIONS / HTTP/1.1" 200 - "-" "Microsoft-WebDAV-MiniRedir/5.1.2600" 192.168.0.148 - - [01/Feb/2007:21:28:57 -0800] "PROPFIND /ccc HTTP/1.1" 207 779 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600" 192.168.0.148 - - [01/Feb/2007:21:28:57 -0800] "PROPFIND /ccc/Client%20A.one HTTP/1.1" 404 357 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600" 192.168.0.148 - - [01/Feb/2007:21:28:57 -0800] "PROPFIND /ccc HTTP/1.1" 207 779 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600" 192.168.0.148 - - [01/Feb/2007:21:28:57 -0800] "PUT /ccc/Client%20A.one HTTP/1.1" 403 361 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600" 192.168.0.148 - - [01/Feb/2007:21:28:57 -0800] "PROPFIND /ccc/Client%20A.one HTTP/1.1" 404 357 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600" 192.168.0.148 - - [01/Feb/2007:21:28:57 -0800] "PROPFIND /ccc HTTP/1.1" 207 779 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600" 192.168.0.148 - - [01/Feb/2007:21:28:57 -0800] "PUT /ccc/Client%20A.one HTTP/1.1" 403 361 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600" 192.168.0.148 - - [01/Feb/2007:21:28:57 -0800] "PROPFIND /ccc/Client%20A.one HTTP/1.1" 404 357 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600" 192.168.0.148 - - [01/Feb/2007:21:28:57 -0800] "PROPFIND /ccc HTTP/1.1" 207 779 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600" 192.168.0.148 - - [01/Feb/2007:21:28:57 -0800] "PUT /ccc/Client%20A.one HTTP/1.1" 403 361 "-" "Microsoft-WebDAV-MiniRedir/5.1.2600" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "HEAD /ccc HTTP/1.1" 301 - "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "MKCOL /ccc HTTP/1.1" 405 370 "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "PROPFIND / HTTP/1.1" 207 6474 "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "HEAD /ccc HTTP/1.1" 301 - "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "MKCOL /ccc HTTP/1.1" 405 370 "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "HEAD /ccc/ HTTP/1.1" 200 - "-" "Microsoft Office Existence Discovery" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "LOCK /ccc/Client%20A.one HTTP/1.1" 200 420 "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "PUT /ccc/Client%20A.one HTTP/1.1" 201 335 "-" "Microsoft Data Access Internet Publishing Provider DAV" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "OPTIONS /ccc/ HTTP/1.1" 200 - "-" "Microsoft Office Protocol Discovery" 192.168.0.148 - jborlik [01/Feb/2007:21:28:57 -0800] "HEAD /ccc/Client%20A.one HTTP/1.1" 200 - "-" "Microsoft Office Existence Discovery"
If you look very closely at the log above, you will see OneNote make its OPTIONS and PROPFIND requests, succeed, and move on. It tries a number of other things over the unsecured URL, including PUT, but it tries again over HTTP (and succeeds). (I tried just allowing OPTIONS over HTTP and disallowing PROPFIND, but that failed.)
With this, I think that things are up and running. It seemed to sync and work correctly. When I restarted OneNote, it reprompted me for the site login (using the IE authentication box), which is fine. I did notice that I couldn't rename sections, as OneNote tried to do a (disallowed) MOVE request over the unsecured HTTP, and never tried again over HTTPS. Ah well… MOVE is a bit more dangerous, so I'll leave that one turned off. I'll just have to be very careful what I name sections when I first make them.
A person might be tempted to say that this was sloppy coding on the OneNote team's part. I don't believe that, though… My guess is that there are many different site setups, and many different WebDAV servers out there, and they tried to make OneNote as compatible as possible. Should they break SharePoint, or Apple iDisk, in order for this to work from my particular setup of Apache? Someone will end up unhappy no matter what they choose. There is a way for me to work around the problem, at least. In the end, I'm glad that the functionality is there, and I salute the OneNote team for a product that makes me more productive.
Update 1/21/2007: Dan Escapa's must-read OneNote blog has some additional information regarding this issue. His solution, from David Tse of Microsoft, is more or less what I did above. Mr. Tse suggests that it doesn't work for Vista, although his solution didn't involve SSL encryption (as far as I could tell), just authentication over plain-old HTTP.
Comments
Thank you for the info. I'm a OneNote junkie that really needs to have it stored in a secure location.
I'm toying with the idea of just setting up a personal Wikipedia site to use instead of OneNote.
Posted by: Dave Fourputt | February 5, 2007 06:12 PM
Apparently the issue is more complex, e.g. it depends on authentication method used. If you use NTLM it works (either over HTTP or HTTPS), but if you use Basic auth, it won't, no matter if you use Apache or Sharepoint (IIS). Tried all possible combinations, in my opinion WAN-wide sync in OneNote is just broken if you want any reasonable security and convinience.
Posted by: Haim G. | February 6, 2007 08:16 AM
Has anyone made any progress with this issue? Haim, you comment regarding NTLM sounds very interesting. I wonder if it is possible to use this type of authentication using Apache and Samba. Has anyone tried this?
Posted by: Jake | February 19, 2007 07:54 PM
Very interesting observations. I have just received access to my corporate SharePoint server's IIS logs so I can check on this same issue. For me, the manifestation is unexpected "Access denied" errors on sections that I'm in the middle of using. I believe Haim G. (two comments up) has an interesting point -- I take my laptop from the office home, and in the office I often roam from wired to wireless (which is secured to our internal network via PEAP); perhaps my authentication changes from NTLM to Basic (SSL)? Arguing in favor of that is the fact that I never experience the problem on my home machine, only my laptop which is part of our domain.
Anyway, I will comment again if I figure it out.
Posted by: sbreck | February 20, 2007 05:11 PM
Looking forward to your research sbreck.
Posted by: Jake | February 21, 2007 10:07 PM
Hmm. I saw a lot of 401.1 and 401.2 errors in my logs and began finding that exiting OneNote and any IE window open to my SharePoint site, then launching IE, then launching the specific section file (but answering "Notebook" to the question OneNote asked) would sometimes fix the problem.
A day or two later, to fix a problem with a web app, I was asked to go to the Advanced tab of Internet Options, and click both Restore Advanced Settings and Reset. This I did, and it fixed the problem with the web app. Then, to my amazement, the "OneNote needs a password to sync some of your notebooks" message began appearing in certain sections. Clicking that message prompted for a password (sometimes twice) but it always synced the notebook. I tempt fate by typing this, but since I reset my IE7 settings, I have successfully been syncing my notebooks with SharePoint in the office and outside for days. This has never happened before. Maybe my IE7 settings had some sort of corrupt or improper security setting for my SharePoint server?
Posted by: sbreck | February 28, 2007 08:49 PM
I don't know if you'll still be tracking comments to this post, but I've been trying to deal with the same problem. A WebDAV share via Apache, providing a central OneNote store for multiple PCs, some on Vista, some on XP.
What you suggest works, except that the HEAD requests that come from the "Microsoft Office Existence Discovery" agent also need to be allowed. Apparently this wasn't an issue for you and I'm wondering why. I'm also wondering if HEAD is unsafe to allow via HTTP. Any insight?
Posted by: mwtb | May 13, 2008 12:46 AM