Pages

Thursday, March 12, 2020

Einstein Analytics: Precision and Scale

Precision and Scale are important and required for computeExpression node that returns Numeric in Dataflow, otherwise, your dataflow rum will fail.

For numeric, as per this article External Data Metadata Format Reference
  • precision: the maximum number of digits in a numeric value, includes all numbers to the left and to the right of the decimal point (but excludes the decimal point character). Value can be up to 18.
  • scale: the number of digits to the right of the decimal point in a numeric value, must be less than the precision value.

But in short:
  precision: must be 1 - 18
  scale: must be 0 - 17 and less than the precision value


Let us see how this works in reality. I'll do a few same calculations on computeExpression, but with different precision and scale, the formula is A/B for all calculations, here is the result:



Calc_10_5 mean, precision = 10 and scale = 5, and etc. At a glance, you may think that all decimal points do not exist, this is incorrect as you need to "format numbers" on the widget or metadata.

For this blog testing, I set 5 digits decimal point:



Here is the result after all fields set with 5 decimal points:



From the above table, "scale" shows the difference in the calculation result, the result will be round up or round down based on the decimal point defined in the scale.

Notice that decimal point "below 0.5" will be round down, while "0.5 and above" will be round up. But, if scale = 0, all decimal points will be round down, see calc_10_0.



Reference: External Data Metadata Format Reference




8 comments:

  1. Thanks a lot Johan. I have a doubdt cant we have text field having character more than 255.
    or how can we achieve it

    ReplyDelete
    Replies
    1. do you mean text field in general in TCRM?

      Delete
    2. Yes Johan, we have a 'Opp description' field in compute expression, which is concatenation of multiple descriptions. The thing is it exceeds 255 char limit, but we want whole description in TCRM dashboard or Report

      Delete
    3. You can update the Precision up to 32000

      Delete
    4. And how exactly we change precision on a dimension? In Analytics studio there is no option available and in Dataset created from CSV the text is already truncated....

      Delete
    5. it is on digest and CE/CR node

      Delete
  2. Even I see in the documentation its mentioned that 32000 char is supported but how to change it?
    precision Number Yes (for Numeric values)
    The maximum number of digits in a numeric value, or the length of a text value.

    For numeric values: Includes all numbers to the left and to the right of the decimal point (but excludes the decimal point character). Value can be up to 18.

    For text values: Value defaults to 255 characters, but can be up to 32,000 characters.

    ReplyDelete

Page-level ad