Jump to content

hex inconsistancy


p.m.berger
 Share

Recommended Posts

In react, if I issue the command:

NO3- 0x3f1a36e2eb1c232c mol 

which should be 1e-4 mol NO3-. Instead it comes out as 1e-3. If I save the file in hex format the NO3- line is

NO3-         = 0x3f50624dd2f197f8 mol

which equals 1e-3 mol NO3-. So it seems as the input is being read incorrectly. Bug?

Thanks,

Peter

Link to comment
Share on other sites

  • 4 weeks later...

It seems there isn't a work around with using standard numbers or scientific notation correct? 

NO3- mol 9.99999999998440423e-05

and 

NO3- mol 0.0000999999999998440423

comes out as

0.001 instead of 0.0001. I'm playing around with the GWB libraries to model multicomponent diffusion so I need the high precision.

Thanks,

Peter

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

×
×
  • Create New...