Created Date: August 24,2016

Created by: OkiePC

Working on a system with a Click C0-10DD1E-D which communicates to a Phoenix Contact TWE radio modem using Modbus TCP/IP. The Phoenix Radio Modem controls I/O at 5 slave radios. I have the master radio and one slave set up on my desk.

The I/O Modbus addresses for the slaves are not in nice tidy groups (using one combo module per slave). So there is a message to read three radio status words, a message to read two coils, a message to read the analog input, a message to write the two output coils, and a message to write the analog output. This ends up making for 5 messages per node plus one for the master radio status words. One node is just reading the analog. To make a short story long there are 24 messages in total.

I was chugging along just fine with my message structure after proving I had it all working right for the master and the first slave. Then things went south when I added the messages (SEND and RECEIVE) for all the missing nodes.

I found that after the first RECEIVE command would error, using that error bit to trigger the next node message, that message rung should be operating but never gets done or errors. I went back and added SENDING and RECEIVING bits to all 24 messages and that did not reveal anything new.

What is weird is that I can unplug the Ethernet cable to the master radio and cause all the messages to error and then the program successfully fails all the way through. When I plug it back in, the situation goes back to where it will hang after the first error and stall out there. I did take steps to try to reduce the potential for message errors by skipping the I/O reads and writes for radios that fail to return status information. I also added a compare instruction to reset my message timer (at the beginning of the subroutine to monitor how long the round robin takes) and this at least lets the logic start over after a certain amount of time.

This is no good in the field, however, because if Node 1 is missing, I still want to be able to read nodes 2 through 5 and repeat.

Monitoring the SC bits for Port 1 reveals nothing useful. All message errors result in SC91 being true (Port 1 Error Flag) and SC90 is always true (Port 1 Ready).

I tried putting all this logic in the main routine instead of a subroutine and got the same result.

Finally, I put a self resetting timer in the main routine (5ms On Delay) and only call the subroutine when it is done. This fixed it for me intermittently, but it broke my message round robin timer logic because the timer accumulator is no longer actually a 1ms timebase. Raising the subroutine call interval to ten milliseconds gave me more consistent message results. So I tried setting the PLC scan time to a fixed interval of 10ms. This did not work.

So I put the timer back to make this thing work for me. I was about to move my message round robin timer logic to the main routine but that might be problematic since the timer done bit is used inside the subroutine to fire off the first message and I want to think through the possibilities. While experimenting, I tried adjusting my main routine timer preset to 10ms. It still works. So I tried 5ms and it still works, but now, my timer in the subroutine does not reset its accumulator between calls. Not only that, but the time is slower...the timebase isn't in seconds any more, it is closer to a 4.99 second timebase... The fact that the timer value in a conditional subroutine cannot be trusted doesn't concern me too much. The Messages hanging up is more of a concern.

So two things are weird with this. First problem is that the message logic hangs and I suspect that there is some internal uh-oh with the way the Ethernet port is handling and/or buffering data.

The way timers are handled inside a conditional subroutine is the 2nd odd thing, but is probably documented somewhere as a normal "feature ".

I will attach the ckp file. You can adjust the value of SC10 (subroutine call frequency) and alter the functionality of the messages in the subroutine. A value of below about 7ms causes the messages to stop. A value of 10ms or higher and it seems pretty reliable. Unless you can hook this thing up to a Modbus device that looks like my master radio, it might be difficult to reproduce.

It might be that I have been too deep into this thing today and missed something simple. All ideas welcome.

Thanks,

Paul

Created Date: August 24,2016

Created by: OkiePC

More experimenting has showed me that I can make my subroutine call unconditional and set the PLC for fixed scan of 50ms and the messages will complete (error or success). This is cleaner and my timer logic to monitor the total round robin cycle works correctly. I am going to adjust the fixed scan interval lower until I find the breaking point. This reminds me of the Controllogix setup for "system overhead timeslice ". Maybe in the Click I am giving the CPU more time to complete other tasks besides solve logic, or maybe I am simply avoiding overrunning the ethernet buffer with this approach. This application is not scan cycle critical so if my scan time is 1000ms it would be okay...

Created Date: August 24,2016

Created by: OkiePC

8ms seems to work 95% of the time today. Yesterday, 10ms didn't want to work. I think I will set it for a fixed interval of 20ms and move on...Maybe this little tidbit of info will help someone in the future. My message cycle completes in 4.5 seconds now even with 4 nodes missing.

EDIT: three hours later, with a scan time fixed at 25ms, my peak message time is 8.975 second and the fastest message loop time is 3.875 seconds with four nodes missing. This proves that it has never "gotten stuck ". As the logic stands now, if it gets stuck, it would reset and record a peak time of 30 seconds.

Created Date: August 24,2016

Created by: OkiePC

See click hardware manual 5th edition 4-34

I am firing my send and receive commands with rising edge contacts. The way I am using the success and error bits to cascade them, only one send or receive can be enabled at any given time.

The help file timing chart does show that there can be some delay between the rung going true and the instruction updating the status bits, but the way I have it programmed with rising edge contacts makes that irrelevant.

Doing the book example type of "roll your own enable bit " interlocking might get hairy for this simple system with five nodes and 24 messages.

For now, all is well by using the fixed scan time of 25ms. With two slaves running and three missing, my total round robin cycle time is under 3 seconds. With all five nodes missing it finishes in about 9 seconds. If I run into further headaches, I will see about rebuilding my comms logic using that method. It has been running all day now without freezing.

I have an upcoming project with 66 slaves and 330 total send and receive instructions. For that one, I plan to use a P2000 PLC.

Created Date: August 29,2016

Created by: OkiePC

I couldn't let it go. I want to find something I am doing wrong. I made a modification today to use set and reset on bits to track the completion (error or success) of each send and receive command. When the bit is set AND the port ready bit is set, I fire the next message and in parallel I reset the firing bit. The next rung examines the success or error bit from that message (rising edge contacts in parallel) to set the next message trigger bit.

Well it's not me. It still won't work if I set the CPU for regular scan mode. Works fine with fixed scan time of 25ms. Works like intended if no messages ever succeed (unplug the master radio Ethernet). So I am reverting back to the simpler version with the 25ms fixed scan and moving on(I promise this time).