OneHotEncoder¶
One-hot encoding.
This transformer will encode every feature it is provided with. If a list or set is provided, this transformer will encode every entry in the list/set. You can apply it to a subset of features by composing it with compose.Select
or compose.SelectType
.
Parameters¶
-
sparse – defaults to
False
Whether or not 0s should be made explicit or not.
Examples¶
Let us first create an example dataset.
>>> from pprint import pprint
>>> import random
>>> import string
>>> random.seed(42)
>>> alphabet = list(string.ascii_lowercase)
>>> X = [
... {
... 'c1': random.choice(alphabet),
... 'c2': random.choice(alphabet),
... }
... for _ in range(4)
... ]
>>> pprint(X)
[{'c1': 'u', 'c2': 'd'},
{'c1': 'a', 'c2': 'x'},
{'c1': 'i', 'c2': 'h'},
{'c1': 'h', 'c2': 'e'}]
We can now apply one-hot encoding. All the provided are one-hot encoded, there is therefore no need to specify which features to encode.
>>> from river import preprocessing
>>> oh = preprocessing.OneHotEncoder(sparse=True)
>>> for x in X:
... oh = oh.learn_one(x)
... pprint(oh.transform_one(x))
{'c1_u': 1, 'c2_d': 1}
{'c1_a': 1, 'c2_x': 1}
{'c1_i': 1, 'c2_h': 1}
{'c1_h': 1, 'c2_e': 1}
The sparse
parameter can be set to False
in order to include the values that are not
present in the output.
>>> oh = preprocessing.OneHotEncoder(sparse=False)
>>> for x in X[:2]:
... oh = oh.learn_one(x)
... pprint(oh.transform_one(x))
{'c1_u': 1, 'c2_d': 1}
{'c1_a': 1, 'c1_u': 0, 'c2_d': 0, 'c2_x': 1}
A subset of the features can be one-hot encoded by using an instance of compose.Select
.
>>> from river import compose
>>> pp = compose.Select('c1') | preprocessing.OneHotEncoder()
>>> for x in X:
... pp = pp.learn_one(x)
... pprint(pp.transform_one(x))
{'c1_u': 1}
{'c1_a': 1, 'c1_u': 0}
{'c1_a': 0, 'c1_i': 1, 'c1_u': 0}
{'c1_a': 0, 'c1_h': 1, 'c1_i': 0, 'c1_u': 0}
You can preserve the c2
feature by using a union:
>>> pp = compose.Select('c1') | preprocessing.OneHotEncoder()
>>> pp += compose.Select('c2')
>>> for x in X:
... pp = pp.learn_one(x)
... pprint(pp.transform_one(x))
{'c1_u': 1, 'c2': 'd'}
{'c1_a': 1, 'c1_u': 0, 'c2': 'x'}
{'c1_a': 0, 'c1_i': 1, 'c1_u': 0, 'c2': 'h'}
{'c1_a': 0, 'c1_h': 1, 'c1_i': 0, 'c1_u': 0, 'c2': 'e'}
Similar to the above examples, we can also pass values as a list. This will one-hot encode all of the entries individually.
>>> X = [{'c1': ['u', 'a'], 'c2': ['d']},
... {'c1': ['a', 'b'], 'c2': ['x']},
... {'c1': ['i'], 'c2': ['h', 'z']},
... {'c1': ['h', 'b'], 'c2': ['e']}]
>>> oh = preprocessing.OneHotEncoder(sparse=True)
>>> for x in X:
... oh = oh.learn_one(x)
... pprint(oh.transform_one(x))
{'c1_a': 1, 'c1_u': 1, 'c2_d': 1}
{'c1_a': 1, 'c1_b': 1, 'c2_x': 1}
{'c1_i': 1, 'c2_h': 1, 'c2_z': 1}
{'c1_b': 1, 'c1_h': 1, 'c2_e': 1}
Processing mini-batches is also possible.
>>> from pprint import pprint
>>> import random
>>> import string
>>> random.seed(42)
>>> alphabet = list(string.ascii_lowercase)
>>> X = pd.DataFrame(
... {
... 'c1': random.choice(alphabet),
... 'c2': random.choice(alphabet),
... }
... for _ in range(4)
... )
>>> X
c1 c2
0 u d
1 a x
2 i h
3 h e
>>> oh = preprocessing.OneHotEncoder(sparse=True)
>>> oh = oh.learn_many(X)
>>> df = oh.transform_many(X)
>>> df.loc[:, sorted(df.columns)]
c1_a c1_h c1_i c1_u c2_d c2_e c2_h c2_x
0 0 0 0 1 1 0 0 0
1 1 0 0 0 0 0 0 1
2 0 0 1 0 0 0 1 0
3 0 1 0 0 0 1 0 0
Keep in mind that ability for sparse transformations is limited in mini-batch case, which might affect speed/memory footprint of your training loop.
Here's a non-sparse example:
>>> oh = preprocessing.OneHotEncoder(sparse=False)
>>> X_init = pd.DataFrame([{'c1': "Oranges", 'c2': "Apples"}])
>>> oh = oh.learn_many(X_init)
>>> oh = oh.learn_many(X)
>>> df = oh.transform_many(X)
>>> df.loc[:, sorted(df.columns)]
c1_Oranges c1_a c1_h c1_i c1_u c2_Apples c2_d c2_e c2_h c2_x
0 0 0 0 0 1 0 1 0 0 0
1 0 1 0 0 0 0 0 0 0 1
2 0 0 0 1 0 0 0 0 1 0
3 0 0 1 0 0 0 0 1 0 0
Methods¶
clone
Return a fresh estimator with the same parameters.
The clone has the same parameters but has not been updated with any data. This works by looking at the parameters from the class signature. Each parameter is either - recursively cloned if it's a River classes. - deep-copied via copy.deepcopy
if not. If the calling object is stochastic (i.e. it accepts a seed parameter) and has not been seeded, then the clone will not be idempotent. Indeed, this method's purpose if simply to return a new instance with the same input parameters.
learn_many
Update with a mini-batch of features.
A lot of transformers don't actually have to do anything during the learn_many
step because they are stateless. For this reason the default behavior of this function is to do nothing. Transformers that however do something during the learn_many
can override this method.
Parameters
- X (pandas.core.frame.DataFrame)
Returns
Transformer: self
learn_one
Update with a set of features x
.
A lot of transformers don't actually have to do anything during the learn_one
step because they are stateless. For this reason the default behavior of this function is to do nothing. Transformers that however do something during the learn_one
can override this method.
Parameters
- x (dict)
Returns
Transformer: self
transform_many
Transform a mini-batch of features.
Parameters
- X (pandas.core.frame.DataFrame)
Returns
DataFrame: A new DataFrame.
transform_one
Transform a set of features x
.
Parameters
- x (dict)
- y – defaults to
None
Returns
dict: The transformed values.