BernoulliNB

Bernoulli Naive Bayes.

Bernoulli Naive Bayes model learns from occurrences between features such as word counts and discrete classes. The input vector must contain positive values, such as counts or TF-IDF values.

Parameters

• alpha – defaults to 1.0

  Additive (Laplace/Lidstone) smoothing parameter (use 0 for no smoothing).

• true_threshold – defaults to 0.0

  Threshold for binarizing (mapping to booleans) features.

Attributes

• class_counts (collections.Counter)

  Number of times each class has been seen.

• feature_counts (collections.defaultdict)

  Total frequencies per feature and class.

Examples

>>> import math
>>> from river import compose
>>> from river import feature_extraction
>>> from river import naive_bayes

>>> docs = [
...     ('Chinese Beijing Chinese', 'yes'),
...     ('Chinese Chinese Shanghai', 'yes'),
...     ('Chinese Macao', 'yes'),
...     ('Tokyo Japan Chinese', 'no')
... ]

>>> model = compose.Pipeline(
...     ('tokenize', feature_extraction.BagOfWords(lowercase=False)),
...     ('nb', naive_bayes.BernoulliNB(alpha=1))
... )

>>> for sentence, label in docs:
...     model = model.learn_one(sentence, label)

>>> model['nb'].p_class('yes')
0.75
>>> model['nb'].p_class('no')
0.25

>>> cp = model['nb'].p_feature_given_class

>>> cp('Chinese', 'yes') == (3 + 1) / (3 + 2)
True

>>> cp('Japan', 'yes') == (0 + 1) / (3 + 2)
True
>>> cp('Tokyo', 'yes') == (0 + 1) / (3 + 2)
True

>>> cp('Beijing', 'yes') == (1 + 1) / (3 + 2)
True
>>> cp('Macao', 'yes') == (1 + 1) / (3 + 2)
True
>>> cp('Shanghai', 'yes') == (1 + 1) / (3 + 2)
True

>>> cp('Chinese', 'no') == (1 + 1) / (1 + 2)
True

>>> cp('Japan', 'no') == (1 + 1) / (1 + 2)
True
>>> cp('Tokyo', 'no') == (1 + 1) / (1 + 2)
True

>>> cp('Beijing', 'no') == (0 + 1) / (1 + 2)
True
>>> cp('Macao', 'no') == (0 + 1) / (1 + 2)
True
>>> cp('Shanghai', 'no') == (0 + 1) / (1 + 2)
True

>>> new_text = 'Chinese Chinese Chinese Tokyo Japan'
>>> tokens = model['tokenize'].transform_one(new_text)
>>> jlh = model['nb'].joint_log_likelihood(tokens)
>>> math.exp(jlh['yes'])
0.005184
>>> math.exp(jlh['no'])
0.021947
>>> model.predict_one(new_text)
'no'

>>> model.predict_proba_one('test')['yes']
0.8831539823829913

You can train the model and make predictions in mini-batch mode using the class methods learn_many and predict_many.

>>> import pandas as pd

>>> docs = [
...     ('Chinese Beijing Chinese', 'yes'),
...     ('Chinese Chinese Shanghai', 'yes'),
...     ('Chinese Macao', 'yes'),
...     ('Tokyo Japan Chinese', 'no')
... ]

>>> docs = pd.DataFrame(docs, columns = ['docs', 'y'])

>>> X, y = docs['docs'], docs['y']

>>> model = compose.Pipeline(
...     ('tokenize', feature_extraction.BagOfWords(lowercase=False)),
...     ('nb', naive_bayes.BernoulliNB(alpha=1))
... )

>>> model = model.learn_many(X, y)

>>> model['nb'].p_class('yes')
0.75

>>> model['nb'].p_class('no')
0.25

>>> cp = model['nb'].p_feature_given_class

>>> cp('Chinese', 'yes') == (3 + 1) / (3 + 2)
True

>>> cp('Japan', 'yes') == (0 + 1) / (3 + 2)
True
>>> cp('Tokyo', 'yes') == (0 + 1) / (3 + 2)
True

>>> cp('Beijing', 'yes') == (1 + 1) / (3 + 2)
True
>>> cp('Macao', 'yes') == (1 + 1) / (3 + 2)
True
>>> cp('Shanghai', 'yes') == (1 + 1) / (3 + 2)
True

>>> cp('Chinese', 'no') == (1 + 1) / (1 + 2)
True

>>> cp('Japan', 'no') == (1 + 1) / (1 + 2)
True
>>> cp('Tokyo', 'no') == (1 + 1) / (1 + 2)
True

>>> cp('Beijing', 'no') == (0 + 1) / (1 + 2)
True
>>> cp('Macao', 'no') == (0 + 1) / (1 + 2)
True
>>> cp('Shanghai', 'no') == (0 + 1) / (1 + 2)
True

>>> unseen_data = pd.Series(
...    ['Taiwanese Taipei', 'Chinese Shanghai'],
...    name = 'docs',
...    index=['river', 'rocks']
... )

>>> model.predict_proba_many(unseen_data)
        no       yes
river  0.116846  0.883154
rocks  0.047269  0.952731

>>> model.predict_many(unseen_data)
river    yes
rocks    yes
dtype: object

>>> unseen_data = pd.Series(
...    ['test'], name = 'docs')

>>> model.predict_proba_many(unseen_data)
    no       yes
0  0.116846  0.883154

>>> model.predict_proba_one('test')
{'no': 0.116846017617009, 'yes': 0.8831539823829913}

>>> bag = feature_extraction.BagOfWords(lowercase=False)

>>> model = naive_bayes.BernoulliNB(alpha=1)

>>> X, y = docs['docs'], docs['y']

>>> X = bag.transform_many(X)

>>> X = pd.DataFrame(X.values, columns=X.columns, dtype=int)

>>> model = model.learn_many(X, y)

>>> unseen_data = bag.transform_many(unseen_data)

>>> unseen_data = pd.DataFrame(unseen_data.values, columns=unseen_data.columns, index=['river'])

>>> model.predict_proba_many(unseen_data)
        no       yes
river  0.116846  0.883154

Methods

clone

Return a fresh estimator with the same parameters.

The clone has the same parameters but has not been updated with any data. This works by looking at the parameters from the class signature. Each parameter is either - recursively cloned if it's a River classes. - deep-copied via copy.deepcopy if not. If the calling object is stochastic (i.e. it accepts a seed parameter) and has not been seeded, then the clone will not be idempotent. Indeed, this method's purpose if simply to return a new instance with the same input parameters.

joint_log_likelihood

Computes the joint log likelihood of input features.

Parameters

• x (dict)

Returns

float: Mapping between classes and joint log likelihood.

joint_log_likelihood_many

Computes the joint log likelihood of input features.

Parameters

• X (pandas.core.frame.DataFrame)

Returns

DataFrame: Input samples joint log likelihood.

learn_many

Updates the model with a term-frequency or TF-IDF pandas dataframe.

Parameters

• X (pandas.core.frame.DataFrame)
• y (pandas.core.series.Series)

Returns

MiniBatchClassifier: self

learn_one

Updates the model with a single observation.

Parameters

• x (dict)
• y (Union[bool, str, int])

Returns

Classifier: self

p_class

p_class_many

p_feature_given_class

predict_many

Predict the outcome for each given sample.

Parameters --------- X A dataframe of features.

Parameters

• X (pandas.core.frame.DataFrame)

Returns

Series: The predicted labels.

predict_one

Predict the label of a set of features x.

Parameters

• x (dict)

Returns

typing.Union[bool, str, int]: The predicted label.

predict_proba_many

Return probabilities using the log-likelihoods in mini-batchs setting.

Parameters

• X (pandas.core.frame.DataFrame)

predict_proba_one

Return probabilities using the log-likelihoods.

Parameters

• x (dict)

References

---

1. The Bernoulli model ↩