Error in Yokogawa Centum CS3000 HIS Communication

M

Thread Starter

mudassir_mehmood

We have Centum CS3000 DCS installed at our plant. Recently, (about 2-3 weeks), we are experiencing a communication problem on our Engineering Workstation. The EWS, all by itself, stops communicating with the network (Vnet) and a Communication Error is triggered in system alarms. Error details are as below.

Error code: 307, Network communication error. The communication error occurs due to failure of the communication card or Control bus of this HIS.

If we look in Windows Event log, we see a VLNIC error (related to the control bus interface card). Details as below.<pre>
Event Type: Error
Event Source: VLNIC
Event Category: None
Event ID: 5005
Date: 1/18/2016
Time: 6:14:30 AM
User: N/A
Computer: HIS0145
Description:
Yokogawa Vnet/VLnet Adapter: Has encountered an internal error and has failed.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 00080000 005c0002 00000000 c000138d
0010: 00000000 00000000 00000000 00000000
0020: 00000000 00000000 00000400 00040001</pre>
Yokogawa documentation says that the last DWord gives the specific error code, but this specific error (00040001) is not among the error codes mentioned in the manual.

This phenomenon is occurring now almost daily and only remedy is to restart the EWS.

Any clues? Thanks.
 
Hi Mudassir,

I'd replace the Vnet card OR swap with one in another HIS. Make sure you get the DIP switches right for the HIS bus address. See where the fault goes - whether it stays with HIS0145 or moves to the other HIS.

Additionally you can try checking the patch version of the Vnet card - as you are using Vnet & CS3000 I suspect your project is fairly old - 15 years? There have been several Vnet card firmware updates over the years.

Cheers,
PB
 
M

mudassir_mehmood

Dear Peter,

Sorry for replying late.

Initially, we chose to ignore the situation, but the problem worsened with time. In the end, the communication failed in less than an hour once the EWS was restarted.

Of course the DIP switch settings were correct, the EWS was in service since 2010 without any problem.

Unable to fix the problem, we decided to replace the Vnet card as well as the EWS machine, as it was now about 6 years old.

We did so, and everything went well. The new EWS is in service and communicating with all FCSes and HISes - Except one HIS, HIS0144.

This is quiet strange. Whenever we try to equalize or make any changes to HIS0144 from System View on EWS, the modification fails and following error is displayed on HIS0144 System Alarms.<pre>
-------------------
Error 511
Database download error
Example: DataBase Download Error From xxxxxxxx File=yyyyyyyy
xxxxxxxx: PC name installed in system generation function.
yyyyyyyy: Database file name
Meaning: File transfer failed.
Cause : System generation function side causes
- Incorrect file address.
- Project not found.
- Transfer source file not found.
- File ID not defined.
- Access transfer destination file cannot be accessed.
- Do no have the target file Read right.

Operation and monitoring function side causes
- Do not have the folder/file Write right.
- File system is full. (Disk FULL)
-------------------</pre>
Note that there are 6 FCSes and 13 HISes on the domain (Domain 1) and all are responding fine, except this HIS0144 which is giving the above error. Before replacing the EWS, this HIS was also fine.

I have checked that shared folder (\\HIS0144\CS1000\) is accessible from EWS with read/write access. We did not do anything on HIS0144, only replaced HIS0145 (EWS) along with new Vnet card.

Do you have any clue?
 
Hello Mudassir, RE: HIS0144 equalization problems, after replacing EWS HIS0145.

As a partial mitigation and for troubleshooting, try to start a pending Equalize from the HIS0144 HIS Setup window (.SH) Equalize tab. The CS3000 user will need adequate privileges.

Any errors or warnings displayed in System View, when you are downloading? Especially about HIS0144 equalization?

Is the HIS0145 operating system different, compared with the replaced machine? Are you using Windows Server 2003 Standard Edition? Specific release, either 2003 SP1 or 2003 R2 (R2 without service packs)?

Is you project stored on HIS0145, or a separate file server?

Comparing HIS0144 with a HIS that works (other than the EWS) [Regardless of no recent changes to HIS0144]:

Are they using the same Equalize Referenced Database in the HIS Setup window Equalize tab?

Is the Ethernet adapter the uppermost in the Network Bindings order (uppermost in the Connections list in advanced settings)?

Firewall settings.

Comparing HIS0144 with a HIS that works (other than the EWS), observe though many differences can coexist:
HIS Software revision
Vnet driver versions
Vnet card model and revisions


From a HIS0144 Windows Explorer window, with CENTUM logged in, do the following timely resolve?: <pre>
\\HIS0145\CS1000Pjt\
\\HIS0145\CS1000\
</pre>
From a HIS0144 command prompt, do the following resolve: <pre>
ping E0145
ping HIS0145
ping 172.17.1.45
</pre>
From a HIS0145 Windows Explorer window, with CENTUM logged in, do the following timely resolve?:
[Probably the case you mentioned was OK] <pre>
\\HIS0144\CS1000\
</pre>
From a HIS0145 command prompt, do the following resolve: <pre>
ping E0144
ping HIS0144
ping 172.17.1.44
</pre>
Aside: If you have been making tag changes since the error occurred, it may be best to reboot HIS0144. As a precaution to avoid problems with some types of FCS communications issues, as this HIS may not be holding a relevant tag list. Though rebooting will give a gap in trends gathered by that HIS.
 
M

mudassir_mehmood

Dear FlashL,

Thanks for your detailed reply. I am glad to inform you that I fixed the problem with your help. You made my day!

All IP addresses were being resolved correctly. Previous EWS had WinXP Pro SP2, new one has SP3, but this was also not the cause of the problem.

The actual problem was access of \\HIS0145\ from HIS0144.

When the previous EWS was installed, I created an admin-level user by the name "user1" (on HIS0145).

Sometime later, when I needed to access \\HIS0145 from HIS0144, I provided user1 credentials and saved the password. That was fine as long as the old EWS was there.

When I created the new EWS, I didn't create the admin user by the name user1, but instead chose "Admin" as the default admin user.

HIS0144 was still trying to get access to HIS0145 by "user1" credentials saved and remembered earlier, and was failing. I just opened \\HIS0145 from Windows Explorer on HIS0144 and when Windows prompted the credentials (with HIS0145\user1 still saved in user name), I provided CENTUM/CENTUM and voila... everything was fixed. HIS0144 equalized with charm. I have been dealing with this DCS for a decade now, but learning process never ends.:)

Everyting is fine now, but there is one minor outstanding problem.

The Electronic Instruction Manual is not accessible from System View.

During installation, I chose to keep the EIM files on disk as recommended. I also installed the "Adobe Reader 7.0" provided on the Centum CS3000 setup CD as usual and also installed the "StoryView" companion app from the setup CD.

However, when I chose to access the manuals, the "StoryView" application quits with the message "Failed to create empty document".

I know this is a rather common error in Centum CS3000 installation, but never figured out the fix. It has something to do with the Adobe Reader/StoryView connectivity.

Note that on the EWS PC, some "Adobe Reader DC" was installed on Windows XP initially. I uninstalled the previous version and proceeded to installed the Adobe Reader 7.0 provided on setup CD. I think the two versions are messed up causing this problem.

Do you know how to fix this?

Once again thanks for your assistance.

Mudassir
 
Hi Mudassir,

I'm glad you got your HIS equalization sorted.

Please note the supported operating systems for Centum CS3000 R3.07 (which is a different story to whether or not it may work):
Windows XP SP3 is _NOT_ a supported configuration
Windows XP SP2 is supported
For your system, the EWS HIS0145 should use:
Windows Server 2003 Standard Edition, either SP1 or R2.

Using a server OS for the project database allows more than 10 simultaneous connections. Luckily you haven't experienced this or noticed side effects in your system.

<b>RE: Instruction Manual giving "Failed to create empty document" errors.</b>

Referring to the control.com discussion guidelines, item 8:
http://control.com/guidelns.php#newt
please start a new thread.

Kind regards,
Flash
 
M

mudassir_mehmood

Dear Flash,

Windows XP SP2 was provided by Yokogawa on all HIS including EWS in the original configuration back in 2005. We never used the server OS. And never experienced anything wrong.

However, your comments regarding SP3 has made me a little nervous. I have installed SP3 on several HIS prepared/replaced in recent years. None gave me problem so far. I even have installed SP3 on another CS3000 EWS which is now in service for almost two years.

Anyways thanks for the guidelines. I will start a new thread.
 
Hi dear.

i would like to share a problem in EWS equalization. If you have any idea please share. I am equalizing my EWS with FCS. the error appears is Different database of my project. Error message number is 513.

I do not know how to equalize the EWS now?
Can you please help in this regard?
 
M

mudassir_mehmood

I'm afraid this is bad news. This error occurs when the timestamp of FCS database doesn't match with that of project database in EWS. Probably the EWS database is outdated. There are several reasons for this. Try replacing the project database in your EWS from the latest project backup available.

The problem is that with this error, you cannot make any changes in FCS. If you don't have the latest project backup to resolve this problem, only other method is to perform an offline download to FCS. But be warned that offline download will STOP the FCS and cause process disruption and you cannot do this while plant is running. So, plan a shutdown and perform offline download in available opportunity. Contact Yokogawa tech support for further help.

Mind that there is no harm running the FCS with this error. The only problem is that you cannot make any modification to your FCS database, even as small as changing the comment of a function block. However, there is no issue if you want to make changes to your HIS (graphics, trends, etc.). So, if you can manage to live with this situation, avoid this activity and wait for a planned shutdown opportunity.
 
M

mudassir_mehmood

If you are facing this problem with a newly prepared HIS, than ignore my previous post.

If you have installed a new HIS this error will occur. The resolution is quite simple. Select the problematic HIS in System View, right-click and perform following actions from Load menu, in the following sequence.

1. Download Project Common Section
2. Download to HIS
3. Download Tag-List

I hope this will resolve the problem.
 
Dear sir,

i have been following your communication on the issues you are facing with the CS3000 SYSTEMS. We are also using CS 3000 with WIN XP SP2 on the workstations. Now we are facing it very difficult to get spare hardware resources. How are you managing the issues of hardware?

Please guide me.

I have DELL T5500 workstations totaling 7 nos.6+1 EWS.
As of now I have opted for DELL Warranty services.

Srinivas J
 
Srinivas J, if you wish to start a new discussion, please start a new thread.

<b>Moderator's note:</b> I didn't make this a new thread since Srinivas J was addressing you and mudassir_mehmood specifically.
 
I'm afraid this is bad news. This error occurs when the timestamp of FCS database doesn't match with that of project database in EWS. Probably the EWS database is outdated. There are several reasons for this. Try replacing the project database in your EWS from the latest project backup available.

The problem is that with this error, you cannot make any changes in FCS. If you don't have the latest project backup to resolve this problem, only other method is to perform an offline download to FCS. But be warned that offline download will STOP the FCS and cause process disruption and you cannot do this while plant is running. So, plan a shutdown and perform offline download in available opportunity. Contact Yokogawa tech support for further help.

Mind that there is no harm
If you are facing this problem with a newly prepared HIS, than ignore my previous post.

If you have installed a new HIS this error will occur. The resolution is quite simple. Select the problematic HIS in System View, right-click and perform following actions from Load menu, in the following sequence.

1. Download Project Common Section
2. Download to HIS
3. Download Tag-List

I hope this will resolve the problem.
Hello
Can i contact you
 
Top