Weird AC Voltages in a PoE Camera System--Followup
Well there certainly was a lot of reaction to my recent post on an interesting PoE (Power over Ethernet) issue I found with some IP cameras. I learned a lot from the responses, and did a bunch of subsequent tests and found out some more info, so I thought I'd post a full follow up here.
In addition to posting here (thanks commenters!), I sent that original blog post to my friend Steve Carlson, who was the Task Force Chair for 802.3af-2003 PoE standards development committee and is still active in IEEE standards development; to the manufacturer of the camera; and to the brilliant members of the Show Control Mailing list (there is a quite deep technical discussion there, and I definitely suggest reading through it).
Several people asked about the integrity of my ground in my test setup, so I investigated that and documented it with this little video:
My ground resistance was less than 1 Ohm, which seems pretty good. This Fluke document (for a ground tester) says that less than 5 is good.
So I moved onto some other issues. Steve suggested putting the camera on a longer cable--the increased resistance, he thought, might cause the PoE circuitry (which is pretty complicated) to go into a different mode. And it did, now giving me about 35 volts DC on the case of the camera. Since the little Netgear FS116P switch had already several ports fail, I decided to go ahead and see what happened if I connected this camera case to my ground lead. This is not some exotic test--this is exactly the same as mounting the camera on the wall touching some building conduit, or putting a C-Clamp on the camera and clamping it to our grounded lighting pipe grid. You can see the results here:
Yep, the port blew out. And, the switch fried the camera. I opened up the switch, but there's nothing obviously damaged or burnt:
You can see here that the ACTi camera has multiple, direct connections to the chassis (which fits directly into an extrusion):
(more pictures here)
I metered out 1/2 an Ohm between the chassis and the negative terminal of the power supply, and, given the construction, it would be pretty much impossible to "float" this chassis from the case. But none of this should matter, because the power supply itself is supposed to be completely isolated from the chassis. According to Steve, within the standard there are only two choices as to how to construct the camera:
1. The PD [Powered Device] uses a DC-DC converter to isolate the PSE [Power Sourcing Equipment] cable-side power supply from the PD’s internal power. In this case the PD designer may choose to have the local DC return connected to the chassis, and may have external connections to other devices. The system is isolated per 802.3 for PoE and MAU requirements.
2. The DC does not use an isolated DC-DC converter and the PD supply return is the same as the PSE cable-side return. This is allowed as long as the PD is completely isolated and has no galvanic connections (chassis, connectors, etc.) to the outside world. About the only thing that meets this requirement is a PD with a non-conductive enclosure and no external connections or other elements that are connected to the power supply.
I had sent the documentation of all my tests to ACTi, the manufacturer of our camera (we now have 20 of these cameras) and Mr. Ando Meritee, Manager of ACTi Product Support Division, much to my surprise, posted his own video, showing that this kind of AC leakage is typical with cameras from many manufacturers:
I have to say that I'm quite impressed that ACTi took the time to respond to my little video! Here's the details of Mr. Meritee's response:
To sum up:
(1) If PSE is not properly isolated then you might get the voltage read on the chassis of any IP camera (Pelco, Axis, Hikvision, ACTi ...). You will also get the same read on the chassis of the PSE device itself!
(2) There are 2 reasons of not having properly isolated PSE: 1. The AC cable of PoE switch power supply does not have a ground pin; 2. the ground pin exists, but it is dummy (nothing is connected to it within the PSE device).
(3) Properly isolated PSE has no voltage reads on the chassis of PSE device itself; Consequently, there is no voltage read on the chassis of the PoE supported IP cameras either.
So, here's my current hypothesis as to what happened. The power supply for the Netgear FS116P switch (which is what I was using when we originally encountered this problem in the theatre) malfunctioned, sending a large DC voltage with significant current out to the ground, which was coupled to the case. When we clamped the camera to the lighting grid, that current found a path to ground through the building steel, and blew out the ports. The switch was on another level of the theatre so we couldn't see the fault behavior of the switch, which is why we blew out so many ports.
When I first tested the situation on the bench (as you saw in the previous entry), I saw a large AC voltage, and mistakenly thought that was the source of the switch-damaging current. It seems to me now that the AC voltage, especially with cheaper PoE switches or injectors, is signifcant, but has pretty much insignificant current, so it won't damage anything. Indeed, I tested this today by grounding out the case of the camera when connected to the little mid-span PoE injector and it seemed to be fine. I also saw no problem when grounding the cameras using the Cisco SGE-2000P Gigabit switches that we are planning to use for the Gravesend Inn. They showed about 1/2 a volt AC (much as the switch in ACTi's video above), but that went very close to zero when I grounded the cameras, so it looks like typical, low-current, high voltage leakage. I have that switch up and running overnight with the Fluke scopemeter running in Min/Max mode to see if any variations come up. (Note: this is obviously not an apples to apples comparison--this Cisco switch costs about $1000 and the little Netgear is about $200.)
So, as the result of my adventures in PoE, here's my basic recommendation if you are using a IP PoE device with a metal case: Spend money on a high quality PoE switch with a solid earth ground, and make sure it has a good ground connection so that this kind of fault condition will be dealt with before you are surprised by blowing things up when you go to install a camera for a show.
It looks like Netgear has a lifetime warranty for this PoE product, so this switch will be going back and we'll see what they say...
There are only two choices here:
1.The PD uses a DC-DC converter to isolate the PSE cable-side power supply from the PD’s internal power. In this case the PD designer may choose to have the local DC return connected to the chassis, and may have external connections to other devices. The system is isolated per 802.3 for PoE and MAU requirements.
2.The DC does not use an isolated DC-DC converter and the PD supply return is the same as the PSE cable-side return. This is allowed as long as the PD is completely isolated and has no galvanic connections (chassis, connectors, etc.) to the outside world. About the only thing that meets this requirement is a PD with a non-conductive enclosure and no external connections or other elements that are connected to the power supply.
Update
on 2011-07-07 22:52 by controlgeek
Members of the show control mailing list had asked for a number of tests of the Netgear switch and camera isolation, and so I shot another video documenting a number of resistance tests:
Looking at the video above and the previous measurements, it seems that there is a very solid connection between the ground terminal and the case (about 1/2 an ohm). But the Ethernet pins seemed isolated from the case, and so I couldn't figure out how the DC was getting through. And then I remembered that the destruction I documented above had a D-Link PoE extractor inline. I'm pretty sure that I saw the exact same 30VDC with the camera directly powered when I did the other testing, but today I could only get that fault current with the PoE extractor inline with the Netgear switch:
(The black wire in all these photos goes to my earth ground documented above)
The voltage seems to be about 48 Volts (the PoE voltage) minus whatever the output voltage of the PoE extractor is set at (this one has 5V and 12V modes, it's in 12V mode above so 48-12 = about 36, when I set it to 5V mode it went up about 7 volts to 40). With the PoE extractor gone, the DC was gone:
However, once again, the Netgear switch seems to have this problem but the Cisco does not:
I let the Cisco switch setup run all night and only saw very slight AC voltage which didn't vary much, and I've never had any problems with those switches, which is what we are using for the Gravesend Inn. So I'm pretty confident that the Cisco switches will work for the show.