adccommunitymod (AutomationDirect) asked a question.

P2-550 stalled after two months

Created Date: May 20,2016

Created By: OkiePC

**** This post has been imported from our legacy forum. Information in this post may be outdated and links contained in the post may no longer work.****

I have an application with three P2000 PLCs talking to each other over radio modems. The Master PLC has one relay output card (P2-08TRS) that is not programmed to do anything at this time. It was going to be used to trigger a Sensaphone dialer, but that idea got nixed. So the output card in the first slot sits there doing nothing, although there is ladder logic keeping the 8 relays turned off. This system ran without any hiccups for about two months, then I got a call from the customer saying it was down "Missing Module ". I got into his system remotely and was able to connect to the PLC. The PLC was stopped with a critical error for the missing module. I had the customer re-seat the module and the PLC still could not "see " it. So I altered the CPU configuration for "No Verification and Enable Hot Swap ", transferred to the PLC and put it back in run mode. I also forced on one of the output relays and asked the operator if he saw its LED light up. He said "no, the card looks dead, I got no lights ". Okay, so we let things hum along like this and about a week later at 6 am (same time as before) the PLC dies again. This time I cannot "see it " from his SCADA PC which talks to it via Ethernet and has Productivity Suite software installed. So I go out there and plug in my laptop to the USB port. Meanwhile, the operator power cycles the PLC and it comes back up running just fine, so I don't get to see the display during the "stalled " condition, but I am showing nothing new in the critical alarms list other than the missing module errors from a week earlier. I try forcing the outputs, re-seating the card, and I get no sign of life from this relay card. I do show a red circle above it in the I/O configuration, but I am not clear on what that means. I look at the module properties and they are blank. It seems that the PLC can "see " the card, but can't really "talk " to it at least not correctly. Just for giggles I decide that I will update the firmware. I had been running PSuite 2.1.1.1, so I upgrade to the latest on my laptop and his SCADA PC. I also downloaded the latest "all in one " firmware package and updated the firmware. I don't recall the exact version it was running, but I did read in the release notes about an issue that could cause the PLC to lose the ability to recognize I/O cards under certain conditions. So what can it hurt? After updating the firmware in the CPU, the card comes back to life, and I can now force the relays in the output card (not wired to anything) to light up and I can see the module properties. I still don't know what brought down the PLC this second time or if I should have any further worries but I do know that updating the firmware fixed at least one problem, so i will just keep checking on it periodically for now. I also added an alarm to the SCADA so it will call someone if the PLC drops out of run mode. I set the Modbus address for CPU In Run to 100100, and the stupid SCADA software cannot see that address. So, I added a line of ladder code to drive an output address 000100, which the SCADA can access. The problem is that the PLC can't turn off that bit if it drops out of run mode, so I added a script to the SCADA to turn off the bit, wait a few seconds, see if it is still off, then alarm. I suppose I could have used one of the relay outputs for this but I really don't like accessing real world I/O addresses from SCADA/HMI devices. In any case, it would be nice if there was a fault routine in the PLC so I could force it to do something after a fault before shutting down. There are gotchas with about any method. One real bright spot was the fact that when I got done with the firmware update quickly and painlessly, I did not have to download the project. I am not sure if this was handled automatically by the PLC or by PSuite, but I like that. I guess I don't actually have any specific questions other than if anyone else has seen this sort of problem and if anyone knows of a "fault routine " or method to take action after a fault similar to what Allen-Bradley offers. Thanks! Paul


  • adccommunitymod (AutomationDirect)

    Created Date: May 20,2016

    Created by: OkiePC

    I have an application with three P2000 PLCs talking to each other over radio modems. The Master PLC has one relay output card (P2-08TRS) that is not programmed to do anything at this time. It was going to be used to trigger a Sensaphone dialer, but that idea got nixed. So the output card in the first slot sits there doing nothing, although there is ladder logic keeping the 8 relays turned off.

    This system ran without any hiccups for about two months, then I got a call from the customer saying it was down "Missing Module ". I got into his system remotely and was able to connect to the PLC. The PLC was stopped with a critical error for the missing module. I had the customer re-seat the module and the PLC still could not "see " it. So I altered the CPU configuration for "No Verification and Enable Hot Swap ", transferred to the PLC and put it back in run mode. I also forced on one of the output relays and asked the operator if he saw its LED light up. He said "no, the card looks dead, I got no lights ".

    Okay, so we let things hum along like this and about a week later at 6 am (same time as before) the PLC dies again. This time I cannot "see it " from his SCADA PC which talks to it via Ethernet and has Productivity Suite software installed.

    So I go out there and plug in my laptop to the USB port. Meanwhile, the operator power cycles the PLC and it comes back up running just fine, so I don't get to see the display during the "stalled " condition, but I am showing nothing new in the critical alarms list other than the missing module errors from a week earlier. I try forcing the outputs, re-seating the card, and I get no sign of life from this relay card. I do show a red circle above it in the I/O configuration, but I am not clear on what that means.

    I look at the module properties and they are blank. It seems that the PLC can "see " the card, but can't really "talk " to it at least not correctly.

    Just for giggles I decide that I will update the firmware. I had been running PSuite 2.1.1.1, so I upgrade to the latest on my laptop and his SCADA PC. I also downloaded the latest "all in one " firmware package and updated the firmware. I don't recall the exact version it was running, but I did read in the release notes about an issue that could cause the PLC to lose the ability to recognize I/O cards under certain conditions. So what can it hurt?

    After updating the firmware in the CPU, the card comes back to life, and I can now force the relays in the output card (not wired to anything) to light up and I can see the module properties.

    I still don't know what brought down the PLC this second time or if I should have any further worries but I do know that updating the firmware fixed at least one problem, so i will just keep checking on it periodically for now. I also added an alarm to the SCADA so it will call someone if the PLC drops out of run mode. I set the Modbus address for CPU In Run to 100100, and the stupid SCADA software cannot see that address. So, I added a line of ladder code to drive an output address 000100, which the SCADA can access. The problem is that the PLC can't turn off that bit if it drops out of run mode, so I added a script to the SCADA to turn off the bit, wait a few seconds, see if it is still off, then alarm.

    I suppose I could have used one of the relay outputs for this but I really don't like accessing real world I/O addresses from SCADA/HMI devices. In any case, it would be nice if there was a fault routine in the PLC so I could force it to do something after a fault before shutting down. There are gotchas with about any method.

    One real bright spot was the fact that when I got done with the firmware update quickly and painlessly, I did not have to download the project. I am not sure if this was handled automatically by the PLC or by PSuite, but I like that.

    I guess I don't actually have any specific questions other than if anyone else has seen this sort of problem and if anyone knows of a "fault routine " or method to take action after a fault similar to what Allen-Bradley offers.

    Thanks!

    Paul

    Expand Post
  • adccommunitymod (AutomationDirect)

    Created Date: June 06,2016

    Created by: OkiePC

    So far so good. No more problems after the firmware update. I did go back and change the way the SCADA detects CPU running by using a spare real world output which will drop out when the CPU stops. A Fault Routine would still be a useful additional feature for other situations.