Fix filling for non numeric dataframes
Summary
In our DataFrameSubhandler, we differentiate between NaN
values that are pushed to the Subhandler and NaN
that is filled between different timestamps.
# Replace NaN with -inf to distinguish between the 'real' NaN and the 'fill' NaN
if pd.isna(_value):
_value = -np.inf
self._data.loc[_timestamp, node.name] = _value
However this is only necessary for numeric values, since they can be filled using a method like interpolate()
. Non-numeric values like Strings cannot be filled and also the datatype of NaN
is float64 and doesn't match e.g. String. So they don't need to be replaced with -np.inf
What is the expected correct behavior?
Don't get warnings from pandas (such as below) when running the tests.
What is the current bug behavior?
Values of every datatype get replaced with -np.inf
when they send NaN
. This emits a FutureWarning, saying that this behaviour is deprecated and will be removed in future versions.
Was there an error message?
test/test_connectors/test_modbus.py::TestConnectorSubscriptions::test_subscribe
/builds/eta-fabrik/public/eta-utility/eta_utility/connectors/sub_handlers.py:513:
FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas.
Value ' something' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
self._data.loc[timestamp, node.name] = value
Steps to reproduce
The warnings occur when running the tests, specifically in:
test/test_connectors/test_modbus.py::TestConnectorSubscriptions::test_subscribe test/test_connectors/test_modbus.py::TestConnectorSubscriptions::test_subscribe_interrupted test/test_connectors/test_opcua.py::TestConnectorSubscriptions::test_subscribe test/test_connectors/test_opcua.py::TestConnectorSubscriptions::test_subscribe_interrupted
Can be tested with e.g.
pytest test/test_connectors/test_modbus.py::TestConnectorSubscriptions::test_subscribe