nalp.models.generators¶
Pre-defined generator architectures.
A package for already-implemented generator models.
- class nalp.models.generators.BiLSTMGenerator(encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64)¶
Bases:
nalp.core.Generator
A BiLSTMGenerator class is the one in charge of a bi-directional Long Short-Term Memory implementation.
References
Hochreiter, Jürgen Schmidhuber. Long short-term memory. Neural computation 9.8 (1997).
- __init__(self, encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64)¶
Initialization method.
- Parameters
encoder – An index to vocabulary encoder.
vocab_size – The size of the vocabulary.
embedding_size – The size of the embedding layer.
hidden_size – The amount of hidden neurons.
- call(self, x: tensorflow.Tensor)¶
Method that holds vital information whenever this class is called.
- Parameters
x – A tensorflow’s tensor holding input data.
- Returns
The same tensor after passing through each defined layer.
- Return type
(tf.Tensor)
- property encoder(self)¶
An encoder generic object.
- class nalp.models.generators.ConvGenerator(input_shape: Optional[Tuple[int, int, int]] = (28, 28, 1), noise_dim: Optional[int] = 100, n_samplings: Optional[int] = 3, alpha: Optional[float] = 0.3)¶
Bases:
nalp.core.Generator
A ConvGenerator class stands for the convolutional generative part of a Generative Adversarial Network.
- __init__(self, input_shape: Optional[Tuple[int, int, int]] = (28, 28, 1), noise_dim: Optional[int] = 100, n_samplings: Optional[int] = 3, alpha: Optional[float] = 0.3)¶
Initialization method.
- Parameters
input_shape – An input shape for the tensor.
noise_dim – Amount of noise dimensions.
n_samplings – Number of upsamplings to perform.
alpha – LeakyReLU activation threshold.
- property alpha(self)¶
LeakyReLU activation threshold.
- call(self, x: tensorflow.Tensor, training: Optional[bool] = True)¶
Method that holds vital information whenever this class is called.
- Parameters
x – A tensorflow’s tensor holding input data.
training – Whether architecture is under training or not.
- Returns
The same tensor after passing through each defined layer.
- Return type
(tf.Tensor)
- property filter_size(self)¶
Initial size of the filter.
- property noise_dim(self)¶
Amount of noise dimensions.
- property sampling_factor(self)¶
Sampling factor used to calculate the upsampling.
- class nalp.models.generators.GRUGenerator(encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64)¶
Bases:
nalp.core.Generator
A GRUGenerator class is the one in charge of a Gated Recurrent Unit implementation.
References
K. Cho, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. Preprint arXiv:1406.1078 (2014).
- __init__(self, encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64)¶
Initialization method.
- Parameters
encoder – An index to vocabulary encoder.
vocab_size – The size of the vocabulary.
embedding_size – The size of the embedding layer.
hidden_size – The amount of hidden neurons.
- call(self, x: tensorflow.Tensor)¶
Method that holds vital information whenever this class is called.
- Parameters
x – A tensorflow’s tensor holding input data.
- Returns
The same tensor after passing through each defined layer.
- Return type
(tf.Tensor)
- property encoder(self)¶
An encoder generic object.
- class nalp.models.generators.GumbelLSTMGenerator(encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64, tau: Optional[float] = 5.0)¶
Bases:
nalp.models.generators.LSTMGenerator
A GumbelLSTMGenerator class is the one in charge of a generative Gumbel-based Long Short-Term Memory implementation.
- __init__(self, encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64, tau: Optional[float] = 5.0)¶
Initialization method.
- Parameters
encoder – An index to vocabulary encoder.
vocab_size – The size of the vocabulary.
embedding_size – The size of the embedding layer.
hidden_size – The amount of hidden neurons.
tau – Gumbel-Softmax temperature parameter.
- call(self, x: tensorflow.Tensor)¶
Method that holds vital information whenever this class is called.
- Parameters
x – A tensorflow’s tensor holding input data.
- Returns
Logit-based predictions, Gumbel-Softmax outputs and predicted token.
- Return type
(Tuple[tf.Tensor, tf.Tensor, tf.Tensor])
- generate_greedy_search(self, start: str, max_length: Optional[int] = 100)¶
Generates text by using greedy search, where the sampled token is always sampled according to the maximum probability.
- Parameters
start – The start string to generate the text.
max_length – Maximum length of generated text.
- Returns
Generated text.
- Return type
(List[str])
- generate_temperature_sampling(self, start: str, max_length: Optional[int] = 100, temperature: Optional[float] = 1.0)¶
Generates text by using temperature sampling, where the sampled token is sampled according to a multinomial/categorical distribution.
- Parameters
start – The start string to generate the text.
max_length – Length of generated text.
temperature – A temperature value to sample the token.
- Returns
Generated text.
- Return type
(List[str])
- generate_top_sampling(self, start: str, max_length: Optional[int] = 100, k: Optional[int] = 0, p: Optional[float] = 0.0)¶
Generates text by using top-k and top-p sampling, where the sampled token is sampled according to the k most likely words distribution, as well as to the maximim cumulative probability p.
- Parameters
start – The start string to generate the text.
max_length – Length of generated text.
k – Indicates the amount of likely words.
p – Maximum cumulative probability to be thresholded.
- Returns
Generated text.
- Return type
(List[str])
- property tau(self)¶
Gumbel-Softmax temperature parameter.
- class nalp.models.generators.GumbelRMCGenerator(encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, n_slots: Optional[int] = 3, n_heads: Optional[int] = 5, head_size: Optional[int] = 10, n_blocks: Optional[int] = 1, n_layers: Optional[int] = 3, tau: Optional[float] = 5)¶
Bases:
nalp.models.generators.RMCGenerator
A GumbelRMCGenerator class is the one in charge of a generative Gumbel-based Relational Memory Core implementation.
- __init__(self, encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, n_slots: Optional[int] = 3, n_heads: Optional[int] = 5, head_size: Optional[int] = 10, n_blocks: Optional[int] = 1, n_layers: Optional[int] = 3, tau: Optional[float] = 5)¶
Initialization method.
- Parameters
encoder – An index to vocabulary encoder.
vocab_size – The size of the vocabulary.
embedding_size – The size of the embedding layer.
n_slots – Number of memory slots.
n_heads – Number of attention heads.
head_size – Size of each attention head.
n_blocks – Number of feed-forward networks.
n_layers – Amout of layers per feed-forward network.
tau – Gumbel-Softmax temperature parameter.
- call(self, x: tensorflow.Tensor)¶
Method that holds vital information whenever this class is called.
- Parameters
x – A tensorflow’s tensor holding input data.
- Returns
Logit-based predictions, Gumbel-Softmax outputs and predicted token.
- Return type
(Tuple[tf.Tensor, tf.Tensor, tf.Tensor])
- generate_greedy_search(self, start: str, max_length: Optional[int] = 100)¶
Generates text by using greedy search, where the sampled token is always sampled according to the maximum probability.
- Parameters
start – The start string to generate the text.
max_length – Maximum length of generated text.
- Returns
Generated text.
- Return type
(List[str])
- generate_temperature_sampling(self, start: str, max_length: Optional[int] = 100, temperature: Optional[float] = 1.0)¶
Generates text by using temperature sampling, where the sampled token is sampled according to a multinomial/categorical distribution.
- Parameters
start – The start string to generate the text.
max_length – Length of generated text.
temperature – A temperature value to sample the token.
- Returns
Generated text.
- Return type
(List[str])
- generate_top_sampling(self, start: str, max_length: Optional[int] = 100, k: Optional[int] = 0, p: Optional[float] = 0.0)¶
Generates text by using top-k and top-p sampling, where the sampled token is sampled according to the k most likely words distribution, as well as to the maximim cumulative probability p.
- Parameters
start – The start string to generate the text.
max_length – Length of generated text.
k – Indicates the amount of likely words.
p – Maximum cumulative probability to be thresholded.
- Returns
Generated text.
- Return type
(List[str])
- property tau(self)¶
Gumbel-Softmax temperature parameter.
- class nalp.models.generators.LSTMGenerator(encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64)¶
Bases:
nalp.core.Generator
A LSTMGenerator class is the one in charge of a Long Short-Term Memory implementation.
References
Hochreiter, Jürgen Schmidhuber. Long short-term memory. Neural computation 9.8 (1997).
- __init__(self, encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64)¶
Initialization method.
- Parameters
encoder – An index to vocabulary encoder.
vocab_size – The size of the vocabulary.
embedding_size – The size of the embedding layer.
hidden_size – The amount of hidden neurons.
- call(self, x: tensorflow.Tensor)¶
Method that holds vital information whenever this class is called.
- Parameters
x – A tensorflow’s tensor holding input data.
- Returns
The same tensor after passing through each defined layer.
- Return type
(tf.Tensor)
- property encoder(self)¶
An encoder generic object.
- class nalp.models.generators.LinearGenerator(input_shape: Optional[Tuple[int, Ellipsis]] = (784,), noise_dim: Optional[int] = 100, n_samplings: Optional[int] = 3, alpha: Optional[float] = 0.01)¶
Bases:
nalp.core.Generator
A LinearGenerator class stands for the linear generative part of a Generative Adversarial Network.
- __init__(self, input_shape: Optional[Tuple[int, Ellipsis]] = (784,), noise_dim: Optional[int] = 100, n_samplings: Optional[int] = 3, alpha: Optional[float] = 0.01)¶
Initialization method.
- Parameters
input_shape – An input shape for the tensor.
noise_dim – Amount of noise dimensions.
n_samplings – Number of upsamplings to perform.
alpha – LeakyReLU activation threshold.
- property alpha(self)¶
LeakyReLU activation threshold.
- call(self, x: tensorflow.Tensor, training: Optional[bool] = True)¶
Method that holds vital information whenever this class is called.
- Parameters
x – A tensorflow’s tensor holding input data.
training – Whether architecture is under training or not.
- Returns
The same tensor after passing through each defined layer.
- Return type
(tf.Tensor)
- property noise_dim(self)¶
Amount of noise dimensions.
- class nalp.models.generators.RMCGenerator(encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, n_slots: Optional[int] = 3, n_heads: Optional[int] = 5, head_size: Optional[int] = 10, n_blocks: Optional[int] = 1, n_layers: Optional[int] = 3)¶
Bases:
nalp.core.Generator
An RMCGenerator class is the one in charge of Relational Recurrent Neural Networks vanilla implementation.
References
A. Santoro, et al. Relational recurrent neural networks. Advances in neural information processing systems (2018).
- __init__(self, encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, n_slots: Optional[int] = 3, n_heads: Optional[int] = 5, head_size: Optional[int] = 10, n_blocks: Optional[int] = 1, n_layers: Optional[int] = 3)¶
Initialization method.
- Parameters
encoder – An index to vocabulary encoder.
vocab_size – The size of the vocabulary.
embedding_size – The size of the embedding layer.
n_slots – Number of memory slots.
n_heads – Number of attention heads.
head_size – Size of each attention head.
n_blocks – Number of feed-forward networks.
n_layers – Amout of layers per feed-forward network.
- call(self, x: tensorflow.Tensor)¶
Method that holds vital information whenever this class is called.
- Parameters
x – A tensorflow’s tensor holding input data.
- Returns
The same tensor after passing through each defined layer.
- Return type
(tf.Tensor)
- property encoder(self)¶
An encoder generic object.
- class nalp.models.generators.RNNGenerator(encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64)¶
Bases:
nalp.core.Generator
An RNNGenerator class is the one in charge of Recurrent Neural Networks vanilla implementation.
References
Elman. Finding structure in time. Cognitive science 14.2 (1990).
- __init__(self, encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[int] = 64)¶
Initialization method.
- Parameters
encoder – An index to vocabulary encoder.
vocab_size – The size of the vocabulary.
embedding_size – The size of the embedding layer.
hidden_size – The amount of hidden neurons.
- call(self, x: tensorflow.Tensor)¶
Method that holds vital information whenever this class is called.
- Parameters
x – A tensorflow’s tensor holding input data.
- Returns
The same tensor after passing through each defined layer.
- Return type
(tf.Tensor)
- property encoder(self)¶
An encoder generic object.
- class nalp.models.generators.StackedRNNGenerator(encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[Tuple[int, Ellipsis]] = (64, 64))¶
Bases:
nalp.core.Generator
A StackedRNNGenerator class is the one in charge of stacked Recurrent Neural Networks implementation.
References
Elman. Finding structure in time. Cognitive science 14.2 (1990).
- __init__(self, encoder: Optional[nalp.encoders.integer.IntegerEncoder] = None, vocab_size: Optional[int] = 1, embedding_size: Optional[int] = 32, hidden_size: Optional[Tuple[int, Ellipsis]] = (64, 64))¶
Initialization method.
- Parameters
encoder – An index to vocabulary encoder.
vocab_size – The size of the vocabulary.
embedding_size – The size of the embedding layer.
hidden_size – Amount of hidden neurons per cell.
- call(self, x: tensorflow.Tensor)¶
Method that holds vital information whenever this class is called.
- Parameters
x – A tensorflow’s tensor holding input data.
- Returns
The same tensor after passing through each defined layer.
- Return type
(tf.Tensor)
- property encoder(self)¶
An encoder generic object.