Devices updated to iOS 9 unable to connect

  • 11
  • Question
  • Updated 9 months ago
  • (Edited)
Our network is using an MC3200 controller running 7.0-8-0 with 50 AP 320's. It was upgraded to 7.x a few weeks ago and has been running very well and addressed most of the issues we had seen with 6.x.

Despite warnings not to, some of our staff and students have gone ahead and updated to iOS 9, and we are seeing examples of devices that simply can't connect to our wireless network at all now - including my own iPhone 6. (which I updated to help diagnose the problem)

The symptoms are that the device which was previously able to connect to the network simply cannot. We have one Open ESS, one using WPA-PSK, and the rest use 802.11x authentication - all are affected, so it doesn't seem to be an authentication issue per se.

On trying to connect to the Open network the device tries for a few seconds and then reports "Unable to join the network ----------". No amount of retrying helps, I can reboot the device, reset network settings, nothing helps.

On an authenticated network it will ask for the password over and over. There are *no* attempts to authenticate showing up when monitoring on the controller with station-log. (no 4 way handshake, no authentication rejection etc)

I captured a console log from the iPhone itself when attempting to connect to the Open network:

Sep 17 13:54:04 Simons-iPhone-6 Preferences[211]
<Warning>: -[APNetworksController isVisible]: _visible(1), _formSheetVisibile(0)
Sep 17 13:54:04 Simons-iPhone-6 Preferences[211] <Warning>: -[APNetworksController _joinNetwork:isOtherNetwork:]: Moving new network to the top: <PSSpecifier 0x13df78260: ID STEnroll, Name 'STEnroll' target <APNetworksController: 0x13e079600>>
Sep 17 13:54:04 Simons-iPhone-6 Preferences[211] <Warning>: -[APNetworksController isVisible]: _visible(1), _formSheetVisibile(0)
Sep 17 13:54:04 Simons-iPhone-6 Preferences[211] <Warning>: -[WiFiNetwork canAttemptJoin]: tlsProfileCanJoin(0), self secure(0), passwordOnlyCanJoin(0), automaticProfileCanJoin(0), isEAPSimOrAKA(0), isHS20Provisioned(0) returning 1,
Sep 17 13:54:04 Simons-iPhone-6 Preferences[211] <Warning>: -[APNetworksController isVisible]: _visible(1), _formSheetVisibile(0)
Sep 17 13:54:06 Simons-iPhone-6 kernel[0] <Notice>: 001868.030401 wlan0.N[698] setDISASSOCIATE@13936: [wifid]:
Sep 17 13:54:06 Simons-iPhone-6 kernel[0] <Notice>: 001868.030698 wlan0.N[699] setASSOCIATE@13610: [wifid]: lowerAuth = AUTHTYPE_OPEN, upperAuth = AUTHTYPE_NONE, key = CIPHER_NONE     .
Sep 17 13:54:06 Simons-iPhone-6 kernel[0] <Notice>: 001868.036568 wlan0.N[700] resetAutoCountry@13446:EnhancedLocaleEnabled: 1, HostCountry:1, fDefaultCountryCode:XZ, fCurrentHostCountryCode:GB
Sep 17 13:54:07 Simons-iPhone-6 kernel[0] <Notice>: 001868.789147 wlan0.N[701] handleAuth@1350:    status = 5 protocol failure: packet not ack'd, reason = 0, flags = 0x0, authtype = 0, addr = 00:0c:e6:02:18:a4
Sep 17 13:54:07 Simons-iPhone-6 kernel[0] <Notice>: 001868.811234 wlan0.N[702] handleAuth@1350:    status = 5 protocol failure: packet not ack'd, reason = 0, flags = 0x0, authtype = 0, addr = 00:0c:e6:02:18:a4
Sep 17 13:54:07 Simons-iPhone-6 kernel[0] <Notice>: 001868.833661 wlan0.N[703] handleAuth@1350:    status = 5 protocol failure: packet not ack'd, reason = 0, flags = 0x0, authtype = 0, addr = 00:0c:e6:02:18:a4
Sep 17 13:54:07 Simons-iPhone-6 kernel[0] <Notice>: 001868.851722 wlan0.N[704] handleAuth@1350:    status = 5 protocol failure: packet not ack'd, reason = 0, flags = 0x0, authtype = 0, addr = 00:0c:e6:02:18:a4
Sep 17 13:54:07 Simons-iPhone-6 kernel[0] <Notice>: 001868.870748 wlan0.N[705] handleAuth@1350:    status = 5 protocol failure: packet not ack'd, reason = 0, flags = 0x0, authtype = 0, addr = 00:0c:e6:02:18:a4
Sep 17 13:54:07 Simons-iPhone-6 kernel[0] <Notice>: 001869.492278 wlan0.N[706] handleSetSSID@1429: interface 0 status = 3 failed due to no matching network found, reason = 0, flags = 0x0, authtype = 0, addr = 00:00:00:00:00:00
Sep 17 13:54:07 Simons-iPhone-6 wifid[42] <Error>: WiFi:[464187247.751459]: Failed to join(-3906 - 0xFFFFF0BE): STEnroll
Sep 17 13:54:11 Simons-iPhone-6 wifid[42] <Error>: WiFi:[464187251.556022]: Network NULL for client Preferences !
Sep 17 13:54:11 Simons-iPhone-6 wifid[42] <Error>: WiFi:[464187251.556517]: Network NULL for client Preferences !
Sep 17 13:54:11 Simons-iPhone-6 wifid[42] <Error>: WiFi:[464187251.557344]: Network NULL for client Preferences !
Sep 17 13:54:11 Simons-iPhone-6 wifid[42] <Error>: WiFi:[464187251.558099]: Network NULL for client Preferences !
Sep 17 13:54:11 Simons-iPhone-6 wifid[42] <Error>: WiFi:[464187251.558921]: Network NULL for client Preferences !

And here is the station-log from the controller for the same session, which shows very little:

2015-Sep-17 13:54:05.111964 | 34:a3:95:7d:7c:ec | Station Assign         | <AID=11>[abgn] removed from <AP=39> ESSID=STEnroll BSSID=00:0c:e6:02:18:a4 Ch=36 reason=Assignment aged out2015-Sep-17 13:54:05.111964 | 34:a3:95:7d:7c:ec | Station Assign         | <AID=42>[abgn] removed from <AP=39> ESSID=STEnroll BSSID=00:0c:e6:02:33:d5 Ch=1 reason=Assignment aged out
2015-Sep-17 13:54:05.155964 | 34:a3:95:7d:7c:ec | Station Assign         | Sent deauth remove to <AP=39> ESSID=STEnroll Ch=1 BSSID=00:0c:e6:02:33:d5 reason=Forced removal for sync with AP
2015-Sep-17 13:54:05.155964 | 34:a3:95:7d:7c:ec | Station Assign         | Sent deauth remove to <AP=39> ESSID=STEnroll Ch=36 BSSID=00:0c:e6:02:18:a4 reason=Forced removal for sync with AP
2015-Sep-17 13:54:05.162964 | 34:a3:95:7d:7c:ec | Station Assign         | <AID=11>[abgn] removed from <AP=39> ESSID=STEnroll BSSID=00:0c:e6:02:18:a4 Ch=36 reason=Assignment aged out
2015-Sep-17 13:54:05.162964 | 34:a3:95:7d:7c:ec | Station Assign         | <AID=42>[abgn] removed from <AP=39> ESSID=STEnroll BSSID=00:0c:e6:02:33:d5 Ch=1 reason=Assignment aged out
2015-Sep-17 13:54:05.213964 | 34:a3:95:7d:7c:ec | Station Assign         | <AID=42>[abgn] removed from <AP=39> ESSID=STEnroll BSSID=00:0c:e6:02:33:d5 Ch=1 reason=Assignment aged out
2015-Sep-17 13:54:06.540964 | 34:a3:95:7d:7c:ec | Mac Filtering          | Mac not in deny list - accept client
2015-Sep-17 13:54:06.540964 | 34:a3:95:7d:7c:ec | Station Assign         | <AID=20>[bgn] assigned to <AP=39> ESSID=STEnroll BSSID=00:0c:e6:02:33:d5 Ch=1 reason=Station probed
2015-Sep-17 13:54:07.167964 | 34:a3:95:7d:7c:ec | Station Assign         | <AID=39>[bgn] assigned to <AP=39> ESSID=STEnroll BSSID=00:0c:e6:02:18:a4 Ch=36 reason=Station probed
2015-Sep-17 13:54:41.117964 | 34:a3:95:7d:7c:ec | Station Assign         | <AID=39>[abgn] removed from <AP=39> ESSID=STEnroll BSSID=00:0c:e6:02:18:a4 Ch=36 reason=Assignment aged out
2015-Sep-17 13:54:41.117964 | 34:a3:95:7d:7c:ec | Station Assign         | <AID=20>[abgn] removed from <AP=39> ESSID=STEnroll BSSID=00:0c:e6:02:33:d5 Ch=1 reason=Assignment aged out

Not all devices seem to be affected - I updated an iPad 4 and it had no trouble connecting, but an iPhone 5S, iPhone 6 and two iPad Air's were all affected.

Having a hunch that it might be Virtual Port related I tried setting up a spare AP with a Native cell WPA-PSK ESS and the iPhone 5S connected to it immediately without problems.

However bizarrely after changing the Test ESS back to virtual port the phone was still able to connect! Further more it now connects to all the other networks that it previously could not.

So I am at a bit of a loss here. I know a lot more information is needed to diagnose this problem but this is more of a quick shout out to see if anyone else has been bitten by iOS 9 yet and has any suggestions, because although very few of our devices are updated yet a high percentage of them seem to be affected so this could be a serious issue if lots of users decide to upgrade. (At the end of the day the devices belong to them so we can only advise them not to update, not force them...)

Thanks

Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes

Posted 1 year ago

  • 11
Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes
For anyone who comes across the same issue, I think I have figured it out myself.

It seems iOS 9 on some devices (not all, perhaps dependent on wifi chipset/firmware in the device ?) is incompatible with "SSID Broadcast Preference" being set to "disabled" on a virtual port network. I have changed it to Till-Association and so far I am able to connect to all our ESS's on the misbehaving devices without problems.

The current default of this setting for new ESS's is actually Till-Assocation, but all our ESS's were created way back when Disabled was the default setting.

Ouch!
(Edited)
Photo of Gustavo D Villarreal

Gustavo D Villarreal

  • 175 Posts
  • 29 Reply Likes
Interesting Simon. I have not seen problems with iOS 9 on the newer AP800s but they don't do VirtualPort, so you might be onto something with the Till-Association setting. I somewhat remember Glenn, here from the forum, recommending enabling till-association on VirtualPort ESSs for iOS devices anyway.
Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes
I might have spoken too soon unfortunately!

While it's true that the device is never able to connect at all with it set to "Disabled", after a bit more extensive testing it seems "Till-Association" is only a partial fix.

Yes I can connect now, but after carrying the device around the campus to different areas and back (through areas of no coverage) I again see failed connection and repeated prompts for the password which eventually allow me to connect.

So it doesn't seem to be reliable. :( We do have band steering enabled (with only a 2 second timeout) so I'm sure that is a contributing factor, however bandsteering was working pretty well with iOS 8.

So if anyone has any further suggestions that would be appreciated. I'd love to drop Virtual Port and use Virtual Cell instead, but our AP's don't support it. Virtual Port causes too many compatibility issues for my liking...
Photo of Glenn De Haes

Glenn De Haes

  • 186 Posts
  • 49 Reply Likes
I had a customer today with AP301 and thus virtual port but on 6.1 that had issues with iPhone 6 upgraded to iOS9 while I don't have any issues on AP332 or AP832 with other customers.

It does indeed seem like Virtual Port related but they had also other issues like not being able to send iMessages over 4G etc so it might well be that iOS9 is not that great on the network side. This wouldn't be the first time...

I don't have any issues myself with iOS9 at the moment.

Would be curious to see a wireless capture of this.

till-association in general makes it "easier" for clients to detect and connect to the network but it holds a risk as well.
In my experience, trying it and seeing what it holds for you is the only way to discover this.
Photo of Service Informatique

Service Informatique

  • 7 Posts
  • 0 Reply Likes
Hello Glenn,
Your customer says hi hehe.
Actually, the 4G thingy was probably a temporary problem on Proximus (telecom operator) side as it now works fine.
However, I can confirm that this issue is affecting all iOS 9 devices that I had within reach today. iPad Airs, iPhone 5S and iPhone 6. I don't know about other devices so I can't say for sure if it's 100% iOS 9-dependant or not.
Everything worked fine before iOS 9. As Simon, connecting to a WPA-PSK ESS will cycle into asking for the key. Connecting to a non-protected ESS (aside from captive portal) will simply result in a "unable to join network".
I opened a ticket and sent system diagnostics logs as well as similar station logs. Unfortunately, they say little if anything at all, imho.
Regards,
Jean-Philippe.
Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes
If it helps, I can confirm that IPhone 5s and iPhone 6 are definitely affected on our network.

However I can also confirm two iPad 4's updated to iOS 9 are not affected, so it doesn't seem to be ALL devices that are affected when updated.

So it is possibly related to a wifi firmware/driver update contained within iOS for specific models of iPad/iPhone.

I'm not certain but I believe the two iPads I saw today that were affected were either Airs or Air 2's.

I can also confirm that Native Cell was unaffected, only Virtual Port was affected for us.
(Edited)
Photo of Service Informatique

Service Informatique

  • 7 Posts
  • 0 Reply Likes
I Wonder what triggered this though and if similar issues may arise with the next Mac OS X upgrade. Not that I'd rush doing it myself but ;-) I never had any device behaving this way before.
Photo of Scott Hughes

Scott Hughes

  • 6 Posts
  • 0 Reply Likes
older 3000 controller with 320s - all iPhones that upgraded to iOS9 can't connect - either Incorrect Password or another prompt for password in a loop.

What I find strange - I have 1 AP out of 72 that seems to work yet they all show same running version.
Photo of Scott Hughes

Scott Hughes

  • 6 Posts
  • 0 Reply Likes
just looked at two phones and both now connected on their own - anyone else noticing that?  Did Meru push out an update?
Photo of Service Informatique

Service Informatique

  • 7 Posts
  • 0 Reply Likes
I doubt they'd push an update without you knowing. That said, if they were, I'd have received the answer and/or fix by now.
Photo of Jan Tore Lund

Jan Tore Lund

  • 2 Posts
  • 0 Reply Likes
We're experiencing the same thing here at our workplace. Only newer models with IOS9 affected, an iphone 4S with IOS9 works fine, but all newer models (5s, 6) can't connect.

The interesting part is that we set up another router (another brand) and connected to that first. When I was already connected to that one, and THEN connected to the Meru network it worked. If I then disconnect from both, and then try to connect to the Meru network it still doesn't work. But seems to work when you're already connected to another wifi? In all other instances it just says incorrect password.

So that's a workaround if you really really need a lil' wifi love.
(Edited)
Photo of Gordon Simpson

Gordon Simpson

  • 4 Posts
  • 1 Reply Like
i found this a temp workaround too, but not to another router, just simply to the hotspot on my ipad. Then the SSID of our office wifi appeared and would connect without the fail error
Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes
Jan - I noticed a similar thing - when a phone connected to another ESS with SSID broadcast preference set to "Till-Assocation" it was then able to connect to the original network it had trouble with where it was set to Disabled - but only for a while. It was not able to connect to that network directly, for example after turning wifi off and on. So this was very sketchy at best as a workaround.

Can you and anyone else seeing this problem let us know whether you are using Native Cell, Virtual Port or Virtual Cell on your network, and what you have SSID Broadcast Preference set to in the ESS configuration ?

If you have it set to Disabled, does changing it to Till-Association allow your devices to connect ?

For us Disabled means the affected devices cannot connect at all (apart from briefly via the connect to another network hack above) but with Till-Association I would say things are about 80% back to normal. Most of the time devices connect fine, although I do still occasionally see prompts for the password when moving between areas.

As we use bandsteering a small amount of this prompting was occurring before iOS 9 anyway (due to the nature of how band steering works, and Apple's buggy iOS 8 wifi implementation that prompts for a password when a connection fails for reasons other than an authentication failure) so it is difficult to separate the two - but the incidence does seem to have increased over what it was on iOS 8.

For now Till-Association is my workaround and I watch this thread with interest - if anyone gets a fix back from Meru please let us know. If I don't make any further progress over the next few days I will probably open up a ticket of my own.
(Edited)
Photo of Jan Tore Lund

Jan Tore Lund

  • 2 Posts
  • 0 Reply Likes
I don't have direct access to the settings, I've just forwarded your awesome info to the network dude we're at the mercy of. But keep us updated if you send in a ticket will you? Thanks.
Photo of Gustavo M

Gustavo M

  • 5 Posts
  • 0 Reply Likes
We have experienced the same issues.IOS9 phones devices can ́t connect and prompt for password in the ESS profiles with captive portal.

Our installation: 52 AP1020i / MC3200 ( v 5.1-90)

We have applied the changes discussed in this thread in a new ESS profile (till-association enabled) and so far it has worked.IOS9 phones can connect again without problems.

Thanks a lot for the help
Photo of Service Informatique

Service Informatique

  • 7 Posts
  • 0 Reply Likes
Personnally, I'd avoid that tweak for now (after all, those who upgraded in a rush were mistaken) and wait for the actual "fix" from MERU. My ticket is still Under review, I assume I'll have an answer by the end of the day.
Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes
I'd point out that "Till-Association" is actually the default when creating a new ESS, ever since early versions of SD 5.3, (Before that the default was Disabled) so I'd hardly call this a "tweak".

Having said that, although it has allowed our devices to connect again, I definitely still intermittently see the same symptoms (not surprising when you think through what Till-Association is actually doing) so I will be opening a Ticket with Meru.

The only other "fix" that I can think of that would completely eliminate the problem would be to go to Native Cell...and that is anything but a quick tweak, and with significant ramifications...

Given the nature of the problem I'm not sure that a true fix will be forthcoming very quickly - from what I have observed so far it seems that there could be a fundamental incompatibility between the behaviour of virtual port, and what iOS 9 (on some devices) considers to be acceptable behaviour, and that Till-Association only really papers over the issue. So I wouldn't expect a true fix to arrive quickly.
(Edited)
Photo of Service Informatique

Service Informatique

  • 7 Posts
  • 0 Reply Likes
I encourage you to open a ticket, this is the only way they will know the issue is not limited to certains sites; and moreover that's how they could eventually gather significant logs and/or explanations. You did extensive testing already, which could lead them into a certain direction. IMHO.
Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes
Yes, I've just opened a ticket.
Photo of Service Informatique

Service Informatique

  • 7 Posts
  • 0 Reply Likes
If you want, you can also refer to my ticket (#4062-8465936).
Photo of Gustavo D Villarreal

Gustavo D Villarreal

  • 175 Posts
  • 29 Reply Likes
I wonder if this has something to do with the MAC randomisation on probes Apple said they would introduce in iOS8 (to counteract user tracking), but was only half-done in very specific scenarios (http://blog.airtightnetworks.com/ios8-mac-randomgate/). Maybe now in iOS9 its more completely implemented and its wreaking havoc on VirtualPort?

This could explain why it only affects newer devices (older devices's radio hardware couldn't do this MAC randomization functions). From what I can understand, it looks like newer Broadcom WiFi chips can do multiple connections simultaneously like time-division, doing probe activity with one or more MACs and then connecting with a different MAC, and even connecting to different networks simultaneously (like AppleTV3s can do, being connected in an ad-hoc manner to an iOS devices without dropping the infrastructure connection).
Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes
I like your thinking - it certainly would fit the symptoms, and also explain why Till-Association is not a 100% fix. I'm not really set up for doing over the air packet capture (not sure if I have an adaptor that supports promiscuous mode) so can't really test to confirm this or not though.
Photo of EIU-Core_Ntwk

EIU-Core_Ntwk

  • 8 Posts
  • 1 Reply Like
Simon, How were you able to collect the console output from your iPhone?
Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes
Hi,

You used to be able to use iPhone Configuration utility from a PC or Mac to capture the system log of an iOS device quite easily but since iOS 8 it can only be done through XCode installed on a Mac. (So not possible on a PC to my knowledge)

You would connect the phone over USB, choose to trust the computer, create a dummy XCode project then go to Window->Devices, click on the device on the side bar, then on the right pane there is a tiny upwards arrow in the bottom left - click that and you will get a live, full window console log from the phone.

I don't know how useful it is to troubleshoot this issue (maybe not at all) but I think it's helpful to get the devices perspective on what it thinks it is doing...
(Edited)
Photo of EIU-Core_Ntwk

EIU-Core_Ntwk

  • 8 Posts
  • 1 Reply Like
Thank you.
Photo of Matthew Panton

Matthew Panton

  • 4 Posts
  • 0 Reply Likes
Hi guys, seeing this issue with ap320s virtual cell as well. I've tried, resetting the network settings on two iPhones, a 5s running 9.1 Beta and a 6 running 9.0. I've tried removing the certificates from the phone that were installed by RADIUS server, nothing has helped, it appears from my very limited testing that the issue is only present on 802.1x authenticated networks, I can connect to all WPA2-PSK networks fine, also what is very odd is that if I'm on a WPA2-PSK network with a good connection then try to connect to my 802.1x network going through the same 1550, it works fine, on the device I removed the certs from they get put back fine and I get a good connection to the 802.1x wireless network. Will raise ticket asap
Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes
I don't think you have the same issue at all Matthew - the problem I was seeing is not related to certificates, nor to encryption type - I was seeing the same problem with any type of network including an open network.
(Edited)
Photo of Matthew Panton

Matthew Panton

  • 4 Posts
  • 0 Reply Likes
Hello Simon,

This is the same issue, I've made a mistake in my description of problem, we're using vport, and the testing of removing the certs and different ways of associating were down to eliminating third party factors. Changing to native cell now to test.
Photo of EIU-Core_Ntwk

EIU-Core_Ntwk

  • 8 Posts
  • 1 Reply Like
We appear to be seeing the issue on our open access network, but only on our AP320s running virtual port. The rest of our deployment is AP1020s running virtual cell and I can't reproduce the issue in those locations.

We are also going to open a support case.
Photo of Scott Hughes

Scott Hughes

  • 6 Posts
  • 0 Reply Likes
Work around - for my personal iPhone running ios 9.....I have enabled Hotspot on my office laptop running Ubuntu.  I connect with the iPhone to the hotspot....I disconnect....I am then able to connect to the Meru 320s without a problem.
Photo of Gordon Simpson

Gordon Simpson

  • 4 Posts
  • 1 Reply Like
Nice one Scott. This is what we have done too.. Set up an iPad in the office and everyone coming in connects to it, then the main Meru Controller SSID appears and allows connections. Its a daily faff but better than the alternative
Photo of Glenn De Haes

Glenn De Haes

  • 186 Posts
  • 49 Reply Likes
Hi guys

Check the field notices on the support portal. This has been identified as an issue with AP300/AP433 in virtual port. The current workaround is to contact support and have them alter the parent beacon until they have worked out a final solution with Apple.
Photo of Glenn De Haes

Glenn De Haes

  • 186 Posts
  • 49 Reply Likes
Got an extra update:


1.       Station sends Authentication Request to Parent BSSID and for obvious reason AP does not respond to Parent BSSID due to which the authentication fails. To Counter this, we have to disable Parent Beaconing through following AP boot script

Radio parentbeacon radio0 never

Radio parentbeacon radio1 never

2.       Disabling Parent beacon now will stop SSID being visible to the station hence we need to make sure SSID broadcast is set to ON and SSID broadcast on Vport is set to Till association ( this ensures the stations sees SSID)

3.       I have tried this three customer sites so far and it has improved the connectivity and customers are monitoring the network.

Hope this helps you guys.

Boot scripts must be placed under ATS/scripts and must be saved as a .scr file. Nowadays, you can use the GUI and go to file management - scripts. You must reboot the APs in order to apply the script.

You can also connect to every AP and just enter these lines and they will be active immediately. Be careful that they will no longer be active after reboot.

Photo of Andres Perez

Andres Perez

  • 41 Posts
  • 4 Reply Likes
I am also having the same issue on AP832i set to VCell so looks like is not only isolated to VPort configuration. Have you heard of any client with VCell and this IOS 9 issue?

Unfortunately we can not enable broadcast on the SSID's
Photo of Gustavo D Villarreal

Gustavo D Villarreal

  • 175 Posts
  • 29 Reply Likes
That is strange Andrés, I have not seen this on AP832i. Thats what we have in our offices. At least, not yet! ....
Photo of Andres Perez

Andres Perez

  • 41 Posts
  • 4 Reply Likes
Which code you are running? im on 6.1-4-2
Photo of Gustavo D Villarreal

Gustavo D Villarreal

  • 175 Posts
  • 29 Reply Likes
This is on the latest 7.0-8-0. I always had random issues with AP832s on 6.all ... 7 is quite stable.
Photo of Andres Perez

Andres Perez

  • 41 Posts
  • 4 Reply Likes
I have been considering upgrading but every time we do something else comes up. I have a Healthcare environment with 4 MC4200 and about 850 AP832 so imagine how much of a headache every time we try to upgrade then revert. 

I believe most of my problems resides on the amount of clients on a daily basis, each controller sees at least 1200 clients at a time all day long.
Photo of Gustavo D Villarreal

Gustavo D Villarreal

  • 175 Posts
  • 29 Reply Likes
Yes, that was the experience in 6.x releases for me too, each upgrade was an adventure... 7 has been very stable and has resolved many issues so far.

I think you mentioned that there is no 'elegant' handoff between controller's zones, so maybe you can upgrade just one controller and see what happens? Better wait for the new patch they are releasing to fix this issue anyway.
Photo of Glenn De Haes

Glenn De Haes

  • 186 Posts
  • 49 Reply Likes
Hi Andres
One of our customers has about 3500 simultaneous clients on every controller (MC4200) and that is not an issue. It takes some process up higher than one would want but they also have all of their profiles still in tunneled mode. This would change once they move to bridge mode which wasn't possible in the past because they need radius assigned VLANs. So 1200 clients shouldn't be your problem.
Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes
Hi Glenn,

Thanks for the info. I already have SSID broadcast enabled on all ESS's (we don't use any hidden networks) and already have SSID broadcast on VPort set to Till-Association, and see devices mostly connecting OK but still see intermittent problems.

I'm trying to understand the pros and cons of additionally disabling the vport parent beacon.

From what you say an iOS 9 device sees the parent beacon broadcasting an SSID it is interested in, will attempt to associate with it, but when it (as expected) can't, it will now just give up and report a password failure, without then attempting to associate with the per-client beacon that vport just started sending for it ?

I can see that turning off the parent beacon would prevent this happening, but how does the device now reliably find the network when SSID broadcast on VPort is only set to Till-Association ?

What if there are no devices connected to the AP at all - then nothing is beaconing any of the SSID's, so unless the device sends an active probe it will never find the network. I'm not sure about iOS but some wireless devices only do passive scanning for non-hidden networks, so this could cause issues with other devices.

What if there are other devices that have been connected for a while so that Till-Association has stopped broadcasting the SSID to any of them ? Same problem where an additional passive scanning device can't find the network ?

In the Vport system, will the AP start sending directed beacons to an un-associated client only if the client sends a probe or association request for an SSID provided by the AP, or will it beacon pre-emptively to any un-associated client that it sees, or to any un-associated client probing for any network ?

I might be wrong but it seems that disabling the parent beacon solves one aspect of the problem only to replace it with another possibly worse problem and potentially collateral damage for other device types.
(Edited)
Photo of Glenn De Haes

Glenn De Haes

  • 186 Posts
  • 49 Reply Likes
Hi Simon

I have quickly tested the workaround.
There are no default beacons as expected.
Every client these days send broadcast probe requests. Once the system sees this, it will respond with a probe response with the client BSSID and the SSID in the probe response. The client can then populate its scan list so you can connect to this network in case you have not connected to it before.

Should you have a passive client, then I expect you to have an issue as the probe request (as always with VPort) is the trigger.
I haven't seen any passive clients for a very long time but you never know.
Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes
Thanks Glenn.

We have had a lot of trouble in the past with Linux netbooks and VPort - the default "Network Manager" in distributions like Ubuntu will not reliably auto connect to a VPort network, I ended up having to install wicd which worked better with vport but still wasn't ideal. I think connman (used on some embedded devices, including OpenElec and OSMC on Raspberry Pi) also has trouble with connecting to virtual port.

So what you're saying is that a modern client will send a generic probe request to see what base stations are there even if it is not currently configured to connect to any networks ? And that Vport will respond even to a probe that is not directed to one of it's SSID's ? (Since the client doesn't know about the SSID yet) So initial population of the network list is not a problem even in a completely quiet environment with no other devices already connected ?

If so I will try making the change to disable parent beacons. If I upload the script to the controller and configure AP's to use it, will the AP reboot automatically after that configuration change or do I need to manually reboot them ? (Ideally in groups so that the entire network is not down at once...)

To be honest I would dump virtual port for virtual cell in a hot second if I could. It has caused us nothing but trouble and we don't use any of its additional functionality, but we are kind of stuck with it since we have AP320's...
(Edited)
Photo of Glenn De Haes

Glenn De Haes

  • 186 Posts
  • 49 Reply Likes
Hi Simon

Yes, a default client will always do ative scanning hence sending a broadcast probe request which will be answered by the system showing the client BSSID and SSID. This allows the client to populate the list of SSIDs for the user as well as respond with an Authentication message to the right BSSID.

If you want, you can test this first with one AP. I assume you have a spare AP?
Make a new ESS profile where you disable 'new AP join ESS' and manually put this AP in the ESS-AP table. You put in VPort and Till-association and then you go to the CLI and type:
Connect <ap-id of new AP>
once you get the prompt of the AP:
AP x > 
you type at this prompt the commands:
radio parentbeacon radio0 never

radio parentbeacon radio1 never

Make sure to use lower case! My bad in the original post.

This will enable the setting instantly and only on this AP. You can then test the clients you want and see how it behaves.

If you want to do it for the others, you must add the boot script to make sure that the settings survives a reboot.

If you want it to be active on all AP at once, you add the boot script and reboot the AP (all at once or one by one, whatever works for you).

If you want to do it without reboot, you connect manually to every AP as I described above and you paste the rules:

radio parentbeacon radio0 never

radio parentbeacon radio1 never

exit

then you can do connect ap 1, paste, arrow up, delte the 1 and put 2, enter, paste, 2 for 3, paste etc.

A bit of work but you can do it without interruptions to the users.

Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes
Hi Glenn,

I was part way through setting up a test similar to what you describe, unfortunately my fears about discovery seemed to be confirmed.

I added a new ESS that nobody else knows about on only one AP and configured one iOS device to connect to that AP (so that the network was saved) and then turned off it's wifi.

I then turned off the parent beacons manually as above. Then I turned the wifi on on a second iOS device (iphone 6, iOS 9) which has not seen the new ESS before, and was not configured to connect to any locally available network.

Sure enough, that device is unable to see the newly created network. I left it for about 3 minutes and it saw other networks but not the newly created one.

I then turned on the wifi on the first device that was preconfigured to connect to the new network, not only did it connect successfully, as soon as it did so the first device still sitting on the network scan list suddenly registered that network as existing, presumably because it saw the Till-Association beacon directed at the other device.

So it appears that an iOS 9 device which is not already configured to connect to the network will not send any probes that result in a virtual port beacon being sent back to it.

When I attempted to connect to the network after it saw it in this manner it took 3 attempts before it would connect. One might be band steering but the second shouldn't be.

Also an affected device doesn't seem to be able to connect after wifi is turned back on unless it sees another devices directed beacon - perhaps due to the MAC address randomisation when scanning, although I am monitoring with inssider and do not see a new beacon being created for any MAC address...it seems almost like the phone is only passively scanning for saved (non-hidden) networks as I originally presumed.

So turning off the parent beacon doesn't look usable to me. Yes you might get away with it in a busy network where there are constantly devices coming and going and one device is able to sniff the Till-Association beacon intended for another device but on a quiet network I see a major chicken and egg problem for new devices that have not yet connected to and saved the network.

So I'm going to hold off on this change. I can't see any easy solution for Meru to fix this either, unless Apple are willing to make changes in iOS 9.

PS what is the default setting for parent beacon so I can change it back without a reboot ? is it always, or no_clients ?
(Edited)
Photo of Glenn De Haes

Glenn De Haes

  • 186 Posts
  • 49 Reply Likes
Hi Simon

I did the same test: dedicated new AP and a new profile that was not known to anyone. I just took my iPhone6 on iOS9 and took a look at the wifi list and the ESS was there. I also had to press it three times (without band steering) but according to the station-log and capture, it actually only connected once.
So this definitely seems an Apple issue.

Apple should indeed change it's code as it should send the Auth to the MAC of the probe response and not from the beacon.

I don't know the default setting for parent_beacon. I don't have access to a system at the moment so check with:
connect <ap_id_of_not_modified_AP>
radio show radio0

you should find the setting somewhere in that output.
Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes
Hi Glenn,

Strange that you were able to see the network with parent beacon disable, I definitely was not able to.

Although Till-Association allows devices to connect again, regardless of whether I disable the parent beacon or not we are just not seeing reliable ability to connect, (prompts for passwords, failing to auto connect from a profile etc) or stay connected after the device sleeps or moves to another location.

While the dust settles and proper fixes are worked on I've decided to take the step of switching our network to Native Cell. If Virtual Cell was an option I would use that of course, but it's not available on our all AP320 network, so my hands are tied.

Yes we lose seamless hand off between AP's, but it's not a use case that matters to us really. Nobody here uses any form of VoIP over Wifi, nobody uses the devices in the hall ways etc...

Over 80% of our wireless devices are student iPads, which are used almost exclusively stationary in classrooms or common areas. When a class finishes a student puts their device to sleep, carries it to the next class then wakes it up again.

What is critical to us is handling a high user density (which Virtual port was struggling to do despite lots of tuning) and *reliable* auto connection of devices onto the network without spurious password prompts or other difficulties, which have been giving us headaches for a while, even on iOS 8.

From what I've read (according to a post from an employee on this forum) disabling Virtual Port does not disable air time fairness so that it's still possible to remain on one channel without excessive collisions so we will see how things go.

First impressions are that getting connected is *much* more reliable. Even though I still have band steering enabled for now I'm not really seeing any issues from that at all. All the devices I've tested are auto connecting very reliably.

Throughput seems to have gone up quite a lot too.

I do notice the difference in roaming behaviour of course, devices sometimes connect to a less than optimal AP at first but usually seem to move themselves to a nearer one within a minute or two.

An interesting thing I noticed while walking the halls with both an iPad 4 (unaffected device) and an iPhone 6 (affected device) with both running a constant ping, is that the client side roaming of the iPhone 6 is *much* better than the older iPad 4.

As you'd expect from our non-contiguous coverage in some areas, the iPad 4 was frequently losing the connection when walking from area to area with an active connection, often taking 20+ seconds to find another AP and get connected again.

Meanwhile the iPhone 6 was almost seamless even in Native Cell - I'd lose maybe one ping as I went from area to area but never lost the connection at all.

I'd speculate that this vastly superior client side roaming capability comes from the newer wifi chips in the later iOS devices that have the ability to connect to multiple devices at once and scan with randomised MAC addresses. Perhaps they use this same ability to actively probe for new AP's without dropping the connection on the existing AP and actually connect to the new AP before disconnecting from the old one ?
Photo of Ingvar Andersson

Ingvar Andersson

  • 1 Post
  • 0 Reply Likes
How do I check the beacon settings? on an AP
Photo of Gustavo D Villarreal

Gustavo D Villarreal

  • 175 Posts
  • 29 Reply Likes
Simon, interesting observations regarding native cell performance. How much would you say the throughput has increased as compared to before iOS8 with VPort?

Meru does recommend Native Cell in certain scenarios, in particular in stationary users with high throughput, like stationary video devices, though also with different channels, so your findings are very interesting.
Photo of EIU-Core_Ntwk

EIU-Core_Ntwk

  • 8 Posts
  • 1 Reply Like
We have tested the fix suggested here and by support. Setting the parentbeacon to never and SSIDBroadcast to till-association. 

This has only partly fixed the issue. Older devices iPad 2, iPad mini2, and iPhone4s seem to work OK now. Newer devices iPhone 5s and 6 still have some issues. The first attempt to connect to the network always fails. After the first failure, if the user attempts to join again it usually works.
Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes
Do you also have band steering enabled, like we do ?

If so, failure to connect the first time, exactly once, is pretty normal, because that is the device taking the "easy" route of trying to connect to the stronger 2.4Ghz signal only to have the AP reject the connection attempt to try to force it to move to 5Ghz.

iOS 8 and later misreports this association rejection as an authentication error, (even though this is well before the authentication stage) thus displaying a password prompt and occasionally forgetting the saved password.

You can find another thread on here from about a year ago with a long grumble from me about that piece of stupidity introduced in iOS 8, which is still present in iOS 9...

Once the network is saved you aren't usually aware of the first failed connection attempt because the device will automatically retry - either moving to 5Ghz, or trying again on 2.4Ghz after the band steering timeout, if you have it set quite short like we do.

(I only use 2 seconds - I only want to statistically move "most" dual band devices to 5Ghz for load distribution and performance, I don't care if some devices are able to skirt around the steering and stay on 2.4Ghz as long as most move)

Occasionally band steering will still result in a "forgotten password" and a prompt though, but I'm reasonably sure this is an iOS 8+ quirk that we just have to live with if we are using band steering.
(Edited)
Photo of EIU-Core_Ntwk

EIU-Core_Ntwk

  • 8 Posts
  • 1 Reply Like
We disabled band steering some time ago. 

It was causing more issues that it was fixing.  In our experience devices from the last few years are already preferring 5Ghz when available. 
Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes
Lucky you - most of our devices are iPad's, all of them dual band, but only a small percentage (maybe 20%) will use 5Ghz left to their own devices... with steering about 70% use 5Ghz and 30% stay on 2.4Ghz.
Photo of EIU-Core_Ntwk

EIU-Core_Ntwk

  • 8 Posts
  • 1 Reply Like
Don't miss understand, I would still like to see more devices use 5GHz on my system. Fortunately we have seen a "natural" shift in 5Ghz usage with about that same total station count. 

My network covers the housing area of a college campus so I see just about every type of device imaginable, some with better wifi chips/drivers than others.

Eventually we gave up on band steering because of issues with lower quality chips/drivers not liking the process.

 
Photo of Gustavo D Villarreal

Gustavo D Villarreal

  • 175 Posts
  • 29 Reply Likes
You need to "naturally" band steer, by making both bands symmetric in coverage. If your network was designed for 2.4GHz coverage and mostly only for coverage, that will be complicated since you have to turn down 2.4GHz considerably, but most of the time there is way too much 2.4GHz coverage anyway.

New devices are also being more smart about deciding for 5GHz not only based on RSSI, but also on possible modulation rates, possible sending quality, security, etc. I believe iOS 9 does this better, but I haven't confirmed.
Photo of Andres Perez

Andres Perez

  • 41 Posts
  • 4 Reply Likes
As i said before, i CAN NOT enable SSID to broadcast so that is not a valid solution for us. The "SSIDBroadcast to till-association" is only available in VPort, we are running VCell.
Photo of Glenn De Haes

Glenn De Haes

  • 186 Posts
  • 49 Reply Likes
Hey Andres

Just wondering why you cannot enable SSID broadcast. Be aware that SSID hiding is not part of the standard and that some devices do not like it at all. Even though it is something 'simple' it is not implemented by everyone in the same way.

The only reason I can imagine that you could benefit from hiding the SSID is that it can help you to prevent the client from roaming too many times between controllers as it will have to probe specifically.
Photo of Andres Perez

Andres Perez

  • 41 Posts
  • 4 Reply Likes
"The only reason I can imagine that you could benefit from hiding the SSID is that it can help you to prevent the client from roaming too many times between controllers as it will have to probe specifically."

You got it right there.

Not an option us.
Photo of JohSm67

JohSm67

  • 1 Post
  • 0 Reply Likes
Hi,

We got an issue with newer iPhones 6+ (Plus) that just got updated to iOS9
Older iphone 5 and Android phones still work!

We have an Guest WIFI network that we try to connect to, but are unable to connect from the phone (Should connect then be redirected to login portal.) the AP's are all using the same BSSID and the Guest WIFI is as i understand not using any WPA2-Enterprise keys. Simply unable to connect after the upgrade.

Have tried alot of different suggestions from the Apple community but non that works or changes this behaviour!

What's strange is that if you first connect to another AP (Open) non Meru and then connect to the Guest WIFI (Meru setup) it works and connects fine fine!

But not if you trying to connect to the Guest WIFI using the MERU setup directly, why is that?

Also i contacted our support/vendor for the Meru environment and they indicated that there might be a workaround from Meru that is to be tested, but maybe they do only refeer to this thred?

/Johan
Photo of Gustavo D Villarreal

Gustavo D Villarreal

  • 175 Posts
  • 29 Reply Likes
The workaround was explained by Glenn on the posts above, it is acknowledged by Meru and they say they are working with Apple for a fix, indicating it is not an issue with Meru. Here is the Field Notice Glenn talked about: https://dl.dropboxusercontent.com/u/286636/Meru-FortinetFieldNotice.pdf
Photo of Sean Dolen

Sean Dolen

  • 1 Post
  • 2 Reply Likes
FYI, Today's 9.0.1 IOS Update does nothing to address the issue.
Photo of Graham Colby

Graham Colby

  • 3 Posts
  • 0 Reply Likes
Hi

We have a site that is experiencing the issue with ios 9, they are running an MC1500 I have created a script but need some help on how to get the script to run on the controller.

Thanks
Graham
Photo of Peter Ward

Peter Ward

  • 3 Posts
  • 0 Reply Likes

We have some clients that are running older firmware on older AP/controller that are seeing issues after updating to IOS 9.01

We had 3 SSID using different security profiles; Mixed PSK, Clear/Captive portal and WPA2 PSK and the device could not connect to any of these. When the client disabled mobile data (for their mobile network) suddenly they could connect to all SSIDs with no issue. Maybe worth a try?

Photo of Simon Byrnand

Simon Byrnand

  • 77 Posts
  • 15 Reply Likes
The majority of our iOS 9 devices are Wifi only iPads, (and hence don't have 3G/4G ability) so I think you'll find this is just a coincidence. The nature of the problem means that even with Till-Association enabled, ability to connect will be somewhat intermittent, and depend on what other devices nearby are doing. (which can cause confusing and misleading results during testing unless you are careful to test on a virgin ESS that nobody else is using)

I switched our network from Virtual Port to Native Cell and no more problems with iOS 9 connectivity. We don't really need the seamless handoff between AP's that Virtual Port provides (all devices are used in stationary scenarios) so until there is a proper 100% fix from Meru I suspect that I will be leaving our network in Native Cell.
(Edited)
Photo of Gordon Simpson

Gordon Simpson

  • 4 Posts
  • 1 Reply Like
@GrahamColby - if you get that assistance, i'd very much appreciate it if you could share it. My contact with Meru essentially told me the MC1500 was end of line and there were not going to be any more firmware/patches for it.... :(
Photo of Gordon Simpson

Gordon Simpson

  • 4 Posts
  • 1 Reply Like
Ok guys I think its fixed - well for our Meru 1500 at least..

this is so simple is embarrassing.. the steps I was asked to do were as follows. I only had to make the first "till association" change and all of our devices sprang to life and connected :)


Workaround: 

1) Please make Sure SSID broadcast is set to ON, on the ESS profile.

2) Please set the SSID broadcast Preference for Vport is set to "Till association". 



Above settings are changed from GUI and does not require AP/controller reboot to take the change in effect. 

(Login to GUI>Configuration>ESS>configure the SSID as per the steps mentioned)



With the above two changes test the IOS9 client connectivity.



If no success Please make following changes on 3-4 Aps in one location and try connecting, 



Identify an access-points for test. 

Access the controller on CLI-SSH (putty). 

•    Type "show ap" to find the apid of the access point which we will use for test. 

•    Type "conn ap <apid>   ex for ap ID 14 type conn ap 14".

•    Type following two commands. 

•    radio parentbeacon radio0 never

•    radio parentbeacon radio1 never

.   Exit




Photo of Pok Ng

Pok Ng

  • 2 Posts
  • 0 Reply Likes
Controller: MC1500 version 5.3-132
APs: 320i and 1020i
SSID: 2 Broadcast and 4 Hidden

Device: iPhone 6, iPhone 6s, iPad Mini, iPad 2
iOS: v9.0.0 and v9.0.1

My iPad 2 with v9.0.0 works without issue. My iPhone 6s cannot connect to any of the 6 SSIDs on both v9.0.0 and v9.0.1

I did not set the 4 Hidden SSID broadcast to ON. I just ran the radio pb command on one 1020i AP. The iPhone 6s connected to this AP without any issue and works with the other APs.

Thank you! 
Photo of Aaron Jones

Aaron Jones

  • 2 Posts
  • 0 Reply Likes
Running ver 4.05 on MC1500, I dont see SSID broadcast Preference. Can we proceed with only the AP parentbeacon commands?
Photo of David Salgado

David Salgado

  • 7 Posts
  • 0 Reply Likes
I have quite a few 1500 running the same version and that option is not available. I upgraded  to version 4.0-165 and made the till-association change and also had users enable the "ask to join networks". I've gotten positive results so far this morning.
Photo of Aaron Jones

Aaron Jones

  • 2 Posts
  • 0 Reply Likes
thanks for the feedback.. appears we are on 4.05-105.. doubtful we'll upgrade but good to know what minimum version we'll need to get installed.
Photo of Pok Ng

Pok Ng

  • 2 Posts
  • 0 Reply Likes
I only ran the "radio pb" cmd and it works. 
Photo of David Salgado

David Salgado

  • 7 Posts
  • 0 Reply Likes
I don't have the option to adjust the parentbeacon on 4.0-165. Aaron, I have some controllers on 4.0-158 and the option for "till-association" is available. 

I have a couple of remote sites on 4.0-105 and was hoping the the parentbeacon adjustment would solve my problems without upgrading.
(Edited)