32 bit floating point measurement to 16 bit scaling


Thread Starter


good night

we have a digital meter that supposed to be connected to a DCS system via modbus. the meter measurements are of the type 32bit floating point number while the DCS system only supports a 16 bit signed int format.
if we take the most significant 16 bits and ignores the least one - for example if the floatint point value is represented as (4167007B) hex we just take the (4167) hex value that means a 51.09% of the full scale value (7FFF)Hex - how much accuracy we get.

Jerry Miille

That is not how a floating point number format works. Do a search on IEEE Floating Point and you will see several explanations.

One possible solution would be to use a Protocol Converter to translate this for you. We sell one that than can read in a floating point number and convert it to either two integers that when multiplied together will get you "close" to the number and another option to convert a float to a long integer. In both cases, you lose everything past the decimal point and with larger numbers (above 65,535) some precision there as well. But, if all you need to do is get "close" that might work for you. You can check here http://www.miille.com/PCds.htm or contact me off list for more information. The links are on the data sheets and web pages.

Jerry Miille
32 bit IEEE floating point numbers don't work like that. You MUST read all 32 bits (ie 2 * 16 bit values) and then convert those two 16 bit words into an integer value. If your DCS has a function for converting IEEE format numbers to integers then great, otherwise google and other posts on this site are your friends.

Accuracy is different than resolution, I'm assuming for now that you mean resolution.

You will get 16 bit resolution if the full signal you are measuring will be scaled/represented by the full 32 bit value. That is to say that if your full scale value that you are interested in is passed back by 0xFFFFFFFF (i.e. 2^32 ) then reading the upper 16 bit word is correct and you throw away the lower 16 bits. If you have control of how the meter value gets scaled than you are all set, make it so that 2^32 is the max value that you care about.

Yikes, I replied earlier regarding throwing away the lower 16 bits, but Jerry has it right. If the meter REALLY sends back a IEEE floating point number than it is more difficult. I'd make sure of this.