Posts

Showing posts with the label normalize

Z score normalizing r dataframe consecutively

Z score normalizing r dataframe consecutively I would like to normalize an R data.frame by computing the z-score using the function scale() . data.frame scale() However, I am not sure whether this approach is subject to "look-ahead bias", which is a finance term for making up features that would not have been known or available during the period being analyzed. These are stock returns, and I want to use this data for a "backtest" (a finance term for validation). I want to make sure that each period's z-score is only using data available up to that point and not the entire series mean and std when computing the z-score. Does anyone know how to perform the calculation for this? Or is there a different approach? Could you provide a sample of your data using reprex::reprex() or dput() . It sounds like you don't want want to standardize columns all at once using all the data, but rather standardize them in periods or chunks. Is this ...

Flatten Json Output

Flatten Json Output How can I turn this data into a flat data frame? I've tried using json_normalize and pivot, but I can't seem to get the format right. This is my desired out put format: SiteName|SiteId|...|CompressorMeterRefID|TankID|TankNumber...|TankID|TankNumber...|TankID|... DateandTime|... Please advise [{'SiteName': 'Reinschmiedt 1-4H (CRP 11)', 'SiteId': 57, 'SiteRefId': 'OK10020', 'Choke': '', 'GasMeter1': 53.25, 'GasMeter1Name': 'Check Meter', 'GasMeter1RefId': '', 'GasMeter2Name': '', 'GasMeter2RefId': '', 'GasMeter3Name': '', 'GasMeter3RefId': '', 'OilMeter1Name': '', 'OilMeter1RefId': '', 'OilMeter2Name': '', 'OilMeter2RefId': '', 'WaterMeter1': 0.0, 'WaterMeter1Name': 'Water Meter', ...