A warm welcome at the new #bytemine-office for a #Zarafa meeting: http://t.co/8WJA3Cx4MO
A lot has been said about iOS6 in the past weeks after its release. One of those things is that users have been reporting so called 'meeting hijacking' or unwanted meeting cancellations from users that have upgraded their iPhone or iPad to iOS6.
Small warning: these issues can be complex, so this blog will be longer than usual. If you want to jump to the solution immediately, be my guest by clicking here (Exchange solution included).
The open source project Z-Push has also started to receive reports about this issue. Strangely, after over a month of problems, neither Microsoft nor Apple have provided a fix or even a workaround for the problem. So we decided to investigate, with the help of some community users, and we quickly found what was going on.
Meeting requests work by having an organizer and one or more attendees. The organizer can do a few things: send a meeting request, update the meeting request, and cancel the meeting request.
For example, Lucy can send a meeting request to Bill and Bob. Bill and Bob can reply back with 'accepted', 'declined' or 'tentative'. The replies of Bill and Bob are completely separate from eachother, and Bill shouldn't be able to influence Bob's calendar, and vice versa. Lucy can decide to change the meeting request, or even cancel it completely, in which case both Bill and Bob are informed of the change or cancellation. That is how it is supposed to work.
The bug in iOS6 causes attendees' iPhones (Bill or Bob in the above example) to send cancellations to all the other attendees, wreaking havoc in those users' calendars, since the meeting is not cancelled in reality, and they have no right cancelling lucy's meeting. This is why it is called a meeting 'hijack'.
We will now describe the problem in iOS6, and the known steps to reproduce it. However, there may be other reproduction steps possible, since there are probably many (strange) ways to produce the same failure mode in iOS6.
The final problem all comes down to this: whenever a meeting item is pushed to the iPhone in which there are no attendees, the iPhone will assume that it is the organizer of that meeting. This is absolutely incorrect, and is the cause of all the problems, as we will see.
The easiest way to show the bug starting to happen is to do something a little strange. Send a meeting request to an iPhone user, but put the user in the Bcc instead of To or Cc. In Outlook, you can do this by adding the iPhone user as an attendee, but changing the attendee type (in the scheduling tab) to 'resource'. This will actually send a meeting request with only one user in the Bcc, with an empty To: and Cc: field.
When this meeting arrives on the iPhone, all looks good:
Now, open up outlook for the iPhone user and just click on the incoming message. Don't accept, reply or anything, just open up the message (either in the preview pane or by double-clicking on it). Outlook has now already processed the item and placed it in your calendar (as 'tentative'). Depending on your settings and version of Exchange, this step is even automatic, in which case you can just skip this step.
This causes the appointment in your calendar that was written by outlook now to be pushed to your iPhone. You suddenly notice the iPhone screen change a little:
You can see now that a number of fields have disappeared. In fact, all the attendees from the previous screen have gone. This is correct in fact, since the only attendee (the iPhone user) was Bcc'd in the message.
However, the iPhone still seems to understand that it is an attendee, so it doesn't look too bad at the moment. But, in fact, in the background, everything is broken already. You don't notice this until you reopen the item (just go back to your inbox and reopen the item):
Did you see the problem yet? Where have the 'maybe', 'decline', 'accept' buttons gone? But...you can now see that the iPhone now regards you as the meeting organizer. How? The 'edit' button at the top. Let's press and see the next screen:
So you can press 'edit' and there is a big button, looming in your face. Nastily, you can now 'delete event'. This is basically saying 'You are the organizer and you can cancel this event'.
Now you might say, OK, so that's that. That's the bug. Please fix it, Apple. Well, yes and no. Sure, this is the bug that is the root cause of the hijackings we have seen, but the steps we just did will actually not be sending a cancellation to anyone.
Why? Because there are no attendees in this meeting, that's why. So it would send a cancellation to all attendees, but there are none, so it doesn't send anything.
So for the actual steps to reproduce real hijacking, we have to find another way of pushing an appointment to the iPhone with no attendees, but still manage to send the cancellation to more than 0 people.
This is where it gets a little more complicated, and you really have to know about how MAPI works, and how delegation works. To reproduce this, you need:
– A meeting organizer, let's call her Lucy
– A meeting attendee, Bill
– Bill's delegate, Mary, that receives a copy of meeting requests (so Bill and Mary both receive the meeting requests)
– A number of other attendee's
The steps are as follows:
1. Send a meeting request from Lucy to Bill and some other attendees (the group Everyone for example)
2. Open ('View') the meeting request on Bill's iPhone when it comes in (almost instant). All is well until now
3. From Mary's inbox in outlook (or OWA, whatever), view the same meeting request. Note here again that this step is automatic in some versions of Exchange, and depends on the Exchange automatic processing settings as well.
Now you need to know about how delegation works. Mary is viewing a forwarded version of the original meeting request that was sent to Bill. As such, (like any forward), the meeting request in Mary's inbox has only one recipient, which is Mary (since she received the forwarded meeting request). Now Mary's Outlook places the forwarded meeting in Bill's calendar. But Mary's outlook knows that she was not really an attendee in the meeting. In fact Mary's Outlook knows nothing about the original attendees. So no attendees are actually written to the item in Bill's calendar. However, a workaround in MAPI (and outlook), writes the original attendees in a special property, allowing outlook to show the attendees in Bill's calendar, but this is not used by ActiveSync. Since ActiveSync now sees an item in Bill's calendar with 0 attendees, that is what is sent to the iPhone.
So now the poisoned item is in Bill's calendar and has been pushed to the iPhone.
4. Decline the item on Bill's phone (yes, decline, since that button is still in view)
The decline is apparently internally translated to a cancellation because of the 0-attendee bug, and is sent to all the recipients of the meeting request.
Result: a meeting cancellation is sent to Bill and the other attendees. There's your cancellation. The original organizer knows nothing of the problems, neither does Bill and nobody shows up at the meeting, although some people are confused about how Bill can cancel Lucy's meeting.
The workaround that the Z-Push team have done is to make sure that we never, ever output a meeting for an attendee with no attendees in it. Even if there are no attendees, you know that the user was invited since he has the item in the calendar, so the workaround is to add himself to the attendee list. The fix sounds simple, but we always have to be weary of unwanted side effects, so this will be in QA for some time before it's released. If you really want, you can patch your own Z-Push or get the latest SVN version from z-push.sf.net.
The real fix is that Apple changes the organizer logic back to what it was, or even better, uses the MeetingStatus field in the WBXML response to find out if it is an organizer. I don't know why this wasn't done in the first place.
If you do any of the following, the chance of it happening will be lower, but not zero:
– Make sure you send meeting requests only to the delegate, not to both the attendee and the delegate. (this is an option in your delegation options in Outlook)
– Turn off the inbox assistant (which has various names in various Exchange versions), which automatically processes incoming meeting requests
– There may be a race condition in Exchange's Inbox assistent putting the calendar in the calendar, compared to when you open the item on the iPhone, so sometimes it works, and sometimes it doesn't.
– We did not test with every version of Outlook and Exchange out there, but it seems pretty solidly reproducible, but possibly there is a combination of factors that can make things better. Please let us know of your findings.
You can do any of the following:
– Wait for Z-Push 2.0.5 RC1 from the Z-Push team, which will contain the fix, and download it here
– Get the patch from berlios SVN, which you can find here
– Update to the latest SVN version
Please remember that we are not releasing the fix as stable yet because we do not know the full impact of the fix yet!
Many thanks to people who provided input for this issue, and thank you for sharing your experiences with iOS6 to make Z-Push even better than it already was. Share your thoughts in the comments below!