nalp.models.generators

Pre-defined generator architectures.

A package for already-implemented generator models.

class nalp.models.generators.BiLSTMGenerator(encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64)

Bases: nalp.core.Generator

A BiLSTMGenerator class is the one in charge of a bi-directional Long Short-Term Memory implementation.

References

19. Hochreiter, Jürgen Schmidhuber. Long short-term memory. Neural computation 9.8 (1997).

__init__(self, encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64)

Initialization method.

Parameters

• encoder – An index to vocabulary encoder.

• vocab_size – The size of the vocabulary.

• embedding_size – The size of the embedding layer.

• hidden_size – The amount of hidden neurons.

call(self, x: tensorflow.Tensor)

Method that holds vital information whenever this class is called.

Parameters

x – A tensorflow’s tensor holding input data.

Returns

The same tensor after passing through each defined layer.

Return type

(tf.Tensor)

property encoder(self)

An encoder generic object.

class nalp.models.generators.ConvGenerator(input_shape: Optional[Tuple[int, int, int]] = (28, 28, 1), noise_dim: Optional[int] = 100, n_samplings: Optional[int] = 3, alpha: Optional[float] = 0.3)

Bases: nalp.core.Generator

A ConvGenerator class stands for the convolutional generative part of a Generative Adversarial Network.

__init__(self, input_shape: Optional[Tuple[int, int, int]] = (28, 28, 1), noise_dim: Optional[int] = 100, n_samplings: Optional[int] = 3, alpha: Optional[float] = 0.3)

Initialization method.

Parameters

• input_shape – An input shape for the tensor.

• noise_dim – Amount of noise dimensions.

• n_samplings – Number of upsamplings to perform.

• alpha – LeakyReLU activation threshold.

property alpha(self)

LeakyReLU activation threshold.

call(self, x: tensorflow.Tensor, training: Optional[bool] = True)

Method that holds vital information whenever this class is called.

Parameters

• x – A tensorflow’s tensor holding input data.

• training – Whether architecture is under training or not.

Returns

The same tensor after passing through each defined layer.

Return type

(tf.Tensor)

property filter_size(self)

Initial size of the filter.

property noise_dim(self)

Amount of noise dimensions.

property sampling_factor(self)

Sampling factor used to calculate the upsampling.

class nalp.models.generators.GRUGenerator(encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64)

Bases: nalp.core.Generator

A GRUGenerator class is the one in charge of a Gated Recurrent Unit implementation.

References

K. Cho, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. Preprint arXiv:1406.1078 (2014).

__init__(self, encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64)

Initialization method.

Parameters

• encoder – An index to vocabulary encoder.

• vocab_size – The size of the vocabulary.

• embedding_size – The size of the embedding layer.

• hidden_size – The amount of hidden neurons.

call(self, x: tensorflow.Tensor)

Method that holds vital information whenever this class is called.

Parameters

x – A tensorflow’s tensor holding input data.

Returns

The same tensor after passing through each defined layer.

Return type

(tf.Tensor)

property encoder(self)

An encoder generic object.

class nalp.models.generators.GumbelLSTMGenerator(encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64, tau: Optional[float] = 5.0)

Bases: nalp.models.generators.LSTMGenerator

A GumbelLSTMGenerator class is the one in charge of a generative Gumbel-based Long Short-Term Memory implementation.

__init__(self, encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64, tau: Optional[float] = 5.0)

Initialization method.

Parameters

• encoder – An index to vocabulary encoder.

• vocab_size – The size of the vocabulary.

• embedding_size – The size of the embedding layer.

• hidden_size – The amount of hidden neurons.

• tau – Gumbel-Softmax temperature parameter.

call(self, x: tensorflow.Tensor)

Method that holds vital information whenever this class is called.

Parameters

x – A tensorflow’s tensor holding input data.

Returns

Logit-based predictions, Gumbel-Softmax outputs and predicted token.

Return type

(Tuple[tf.Tensor, tf.Tensor, tf.Tensor])

generate_greedy_search(self, start: str, max_length: Optional[int] = 100)

Generates text by using greedy search, where the sampled token is always sampled according to the maximum probability.

Parameters

• start – The start string to generate the text.

• max_length – Maximum length of generated text.

Returns

Generated text.

Return type

(List[str])

generate_temperature_sampling(self, start: str, max_length: Optional[int] = 100, temperature: Optional[float] = 1.0)

Generates text by using temperature sampling, where the sampled token is sampled according to a multinomial/categorical distribution.

Parameters

• start – The start string to generate the text.

• max_length – Length of generated text.

• temperature – A temperature value to sample the token.

Returns

Generated text.

Return type

(List[str])

generate_top_sampling(self, start: str, max_length: Optional[int] = 100, k: Optional[int] = 0, p: Optional[float] = 0.0)

Generates text by using top-k and top-p sampling, where the sampled token is sampled according to the k most likely words distribution, as well as to the maximim cumulative probability p.

Parameters

• start – The start string to generate the text.

• max_length – Length of generated text.

• k – Indicates the amount of likely words.

• p – Maximum cumulative probability to be thresholded.

Returns

Generated text.

Return type

(List[str])

property tau(self)

Gumbel-Softmax temperature parameter.

class nalp.models.generators.GumbelRMCGenerator(encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, n_slots: Optional[int] = 3, n_heads: Optional[int] = 5, head_size: Optional[int] = 10, n_blocks: Optional[int] = 1, n_layers: Optional[int] = 3, tau: Optional[float] = 5)

Bases: nalp.models.generators.RMCGenerator

A GumbelRMCGenerator class is the one in charge of a generative Gumbel-based Relational Memory Core implementation.

__init__(self, encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, n_slots: Optional[int] = 3, n_heads: Optional[int] = 5, head_size: Optional[int] = 10, n_blocks: Optional[int] = 1, n_layers: Optional[int] = 3, tau: Optional[float] = 5)

Initialization method.

Parameters

• encoder – An index to vocabulary encoder.

• vocab_size – The size of the vocabulary.

• embedding_size – The size of the embedding layer.

• n_slots – Number of memory slots.

• n_heads – Number of attention heads.

• head_size – Size of each attention head.

• n_blocks – Number of feed-forward networks.

• n_layers – Amout of layers per feed-forward network.

• tau – Gumbel-Softmax temperature parameter.

call(self, x: tensorflow.Tensor)

Method that holds vital information whenever this class is called.

Parameters

x – A tensorflow’s tensor holding input data.

Returns

Logit-based predictions, Gumbel-Softmax outputs and predicted token.

Return type

(Tuple[tf.Tensor, tf.Tensor, tf.Tensor])

generate_greedy_search(self, start: str, max_length: Optional[int] = 100)

Generates text by using greedy search, where the sampled token is always sampled according to the maximum probability.

Parameters

• start – The start string to generate the text.

• max_length – Maximum length of generated text.

Returns

Generated text.

Return type

(List[str])

generate_temperature_sampling(self, start: str, max_length: Optional[int] = 100, temperature: Optional[float] = 1.0)

Generates text by using temperature sampling, where the sampled token is sampled according to a multinomial/categorical distribution.

Parameters

• start – The start string to generate the text.

• max_length – Length of generated text.

• temperature – A temperature value to sample the token.

Returns

Generated text.

Return type

(List[str])

generate_top_sampling(self, start: str, max_length: Optional[int] = 100, k: Optional[int] = 0, p: Optional[float] = 0.0)

Generates text by using top-k and top-p sampling, where the sampled token is sampled according to the k most likely words distribution, as well as to the maximim cumulative probability p.

Parameters

• start – The start string to generate the text.

• max_length – Length of generated text.

• k – Indicates the amount of likely words.

• p – Maximum cumulative probability to be thresholded.

Returns

Generated text.

Return type

(List[str])

property tau(self)

Gumbel-Softmax temperature parameter.

class nalp.models.generators.LSTMGenerator(encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64)

Bases: nalp.core.Generator

A LSTMGenerator class is the one in charge of a Long Short-Term Memory implementation.

References

19. Hochreiter, Jürgen Schmidhuber. Long short-term memory. Neural computation 9.8 (1997).

__init__(self, encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64)

Initialization method.

Parameters

• encoder – An index to vocabulary encoder.

• vocab_size – The size of the vocabulary.

• embedding_size – The size of the embedding layer.

• hidden_size – The amount of hidden neurons.

call(self, x: tensorflow.Tensor)

Method that holds vital information whenever this class is called.

Parameters

x – A tensorflow’s tensor holding input data.

Returns

The same tensor after passing through each defined layer.

Return type

(tf.Tensor)

property encoder(self)

An encoder generic object.

class nalp.models.generators.LinearGenerator(input_shape: Optional[Tuple[int, Ellipsis]] = (784,), noise_dim: Optional[int] = 100, n_samplings: Optional[int] = 3, alpha: Optional[float] = 0.01)

Bases: nalp.core.Generator

A LinearGenerator class stands for the linear generative part of a Generative Adversarial Network.

__init__(self, input_shape: Optional[Tuple[int, Ellipsis]] = (784,), noise_dim: Optional[int] = 100, n_samplings: Optional[int] = 3, alpha: Optional[float] = 0.01)

Initialization method.

Parameters

• input_shape – An input shape for the tensor.

• noise_dim – Amount of noise dimensions.

• n_samplings – Number of upsamplings to perform.

• alpha – LeakyReLU activation threshold.

property alpha(self)

LeakyReLU activation threshold.

call(self, x: tensorflow.Tensor, training: Optional[bool] = True)

Method that holds vital information whenever this class is called.

Parameters

• x – A tensorflow’s tensor holding input data.

• training – Whether architecture is under training or not.

Returns

The same tensor after passing through each defined layer.

Return type

(tf.Tensor)

property noise_dim(self)

Amount of noise dimensions.

class nalp.models.generators.RMCGenerator(encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, n_slots: Optional[int] = 3, n_heads: Optional[int] = 5, head_size: Optional[int] = 10, n_blocks: Optional[int] = 1, n_layers: Optional[int] = 3)

Bases: nalp.core.Generator

An RMCGenerator class is the one in charge of Relational Recurrent Neural Networks vanilla implementation.

References

A. Santoro, et al. Relational recurrent neural networks. Advances in neural information processing systems (2018).

__init__(self, encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, n_slots: Optional[int] = 3, n_heads: Optional[int] = 5, head_size: Optional[int] = 10, n_blocks: Optional[int] = 1, n_layers: Optional[int] = 3)

Initialization method.

Parameters

• encoder – An index to vocabulary encoder.

• vocab_size – The size of the vocabulary.

• embedding_size – The size of the embedding layer.

• n_slots – Number of memory slots.

• n_heads – Number of attention heads.

• head_size – Size of each attention head.

• n_blocks – Number of feed-forward networks.

• n_layers – Amout of layers per feed-forward network.

call(self, x: tensorflow.Tensor)

Method that holds vital information whenever this class is called.

Parameters

x – A tensorflow’s tensor holding input data.

Returns

The same tensor after passing through each defined layer.

Return type

(tf.Tensor)

property encoder(self)

An encoder generic object.

class nalp.models.generators.RNNGenerator(encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64)

Bases: nalp.core.Generator

An RNNGenerator class is the one in charge of Recurrent Neural Networks vanilla implementation.

References

10. Elman. Finding structure in time. Cognitive science 14.2 (1990).

__init__(self, encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64)

Initialization method.

Parameters

• encoder – An index to vocabulary encoder.

• vocab_size – The size of the vocabulary.

• embedding_size – The size of the embedding layer.

• hidden_size – The amount of hidden neurons.

call(self, x: tensorflow.Tensor)

Method that holds vital information whenever this class is called.

Parameters

x – A tensorflow’s tensor holding input data.

Returns

The same tensor after passing through each defined layer.

Return type

(tf.Tensor)

property encoder(self)

An encoder generic object.

class nalp.models.generators.StackedRNNGenerator(encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[Tuple[int, Ellipsis]] = (64, 64))

Bases: nalp.core.Generator

A StackedRNNGenerator class is the one in charge of stacked Recurrent Neural Networks implementation.

References

10. Elman. Finding structure in time. Cognitive science 14.2 (1990).

__init__(self, encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[Tuple[int, Ellipsis]] = (64, 64))

Initialization method.

Parameters

• encoder – An index to vocabulary encoder.

• vocab_size – The size of the vocabulary.

• embedding_size – The size of the embedding layer.

• hidden_size – Amount of hidden neurons per cell.

call(self, x: tensorflow.Tensor)

Method that holds vital information whenever this class is called.

Parameters

x – A tensorflow’s tensor holding input data.

Returns

The same tensor after passing through each defined layer.

Return type

(tf.Tensor)

property encoder(self)

An encoder generic object.