metaperceptron.core package

metaperceptron.core.base_mlp module

class metaperceptron.core.base_mlp.BaseMhaMlp(hidden_layers=(100,), act_names='ELU', dropout_rates=0.2, act_output=None, optim='BaseGA', optim_paras=None, obj_name=None, seed=42, verbose=True)

Bases: BaseMlp

Base class for Metaheuristic-based MLP models that inherit from BaseMlp.

SUPPORTED_OPTIMIZERS

List of supported optimizer names.

Type:

list

SUPPORTED_CLS_OBJECTIVES

Supported objectives for classification tasks.

Type:

dict

SUPPORTED_REG_OBJECTIVES

Supported objectives for regression tasks.

Type:

dict

SUPPORTED_CLS_METRICS

Supported metrics for classification evaluation.

Type:

dict

SUPPORTED_REG_METRICS

Supported metrics for regression evaluation.

Type:

dict

Parameters:

• hidden_layers (tuple, list, int) – The number of neurons in each hidden layer.

• act_names (tuple, list, str) – The name of the activation function to be used.

• dropout_rates (tuple, list, float) – The dropout rate for regularization.

• act_output (any, optional) – Activation function for output layer (default is None).

• optim (str) – Name of the optimization algorithm to be used.

• optim_paras (dict, optional) – Parameters for the optimizer (default is None).

• obj_name (str, optional) – Objective name for the model evaluation (default is None).

• seed (int) – Random seed for reproducibility (default is 42).

• verbose (bool) – Whether to print verbose output during training (default is True).

__init__(hidden_layers, act_names, dropout_rates, act_output, optim, optim_paras, obj_name, seed, verbose):

Initializes the model parameters and configuration.

_set_optimizer(optim, optim_paras):

Sets the optimizer based on the provided name or instance.

build_model():

Builds the model architecture and sets the optimizer and loss function.

_set_lb_ub(lb, ub, n_dims):

Validates and sets the lower and upper bounds for optimization.

objective_function(solution):

Evaluates the fitness function for the given solution.

_fit(data, lb, ub, mode, n_workers, termination, save_population, \*\*kwargs):

Fits the model to the provided data using the optimizer.

SUPPORTED_CLS_OBJECTIVES = {'AS': 'max', 'BSL': 'min', 'CEL': 'min', 'CKS': 'max', 'F1S': 'max', 'F2S': 'max', 'FBS': 'max', 'GINI': 'min', 'GMS': 'max', 'HL': 'min', 'HS': 'max', 'JSI': 'max', 'KLDL': 'min', 'LS': 'max', 'MCC': 'max', 'NPV': 'max', 'PS': 'max', 'ROC-AUC': 'max', 'RS': 'max', 'SS': 'max'}

SUPPORTED_OPTIMIZERS = ['OriginalABC', 'OriginalACOR', 'AugmentedAEO', 'EnhancedAEO', 'ImprovedAEO', 'ModifiedAEO', 'OriginalAEO', 'MGTO', 'OriginalAGTO', 'DevALO', 'OriginalALO', 'OriginalAO', 'OriginalAOA', 'IARO', 'LARO', 'OriginalARO', 'OriginalASO', 'OriginalAVOA', 'OriginalArchOA', 'AdaptiveBA', 'DevBA', 'OriginalBA', 'DevBBO', 'OriginalBBO', 'OriginalBBOA', 'OriginalBES', 'ABFO', 'OriginalBFO', 'OriginalBMO', 'DevBRO', 'OriginalBRO', 'OriginalBSA', 'ImprovedBSO', 'OriginalBSO', 'CleverBookBeesA', 'OriginalBeesA', 'ProbBeesA', 'OriginalCA', 'OriginalCDO', 'OriginalCEM', 'OriginalCGO', 'DevCHIO', 'OriginalCHIO', 'OriginalCOA', 'OCRO', 'OriginalCRO', 'OriginalCSA', 'OriginalCSO', 'OriginalCircleSA', 'OriginalCoatiOA', 'JADE', 'OriginalDE', 'SADE', 'SAP_DE', 'DevDMOA', 'OriginalDMOA', 'OriginalDO', 'DevEFO', 'OriginalEFO', 'OriginalEHO', 'AdaptiveEO', 'ModifiedEO', 'OriginalEO', 'OriginalEOA', 'LevyEP', 'OriginalEP', 'CMA_ES', 'LevyES', 'OriginalES', 'Simple_CMA_ES', 'OriginalESOA', 'OriginalEVO', 'OriginalFA', 'DevFBIO', 'OriginalFBIO', 'OriginalFFA', 'OriginalFFO', 'OriginalFLA', 'DevFOA', 'OriginalFOA', 'WhaleFOA', 'OriginalFOX', 'OriginalFPA', 'BaseGA', 'EliteMultiGA', 'EliteSingleGA', 'MultiGA', 'SingleGA', 'OriginalGBO', 'DevGCO', 'OriginalGCO', 'OriginalGJO', 'OriginalGOA', 'DevGSKA', 'OriginalGSKA', 'Matlab101GTO', 'Matlab102GTO', 'OriginalGTO', 'GWO_WOA', 'IGWO', 'OriginalGWO', 'RW_GWO', 'OriginalHBA', 'OriginalHBO', 'OriginalHC', 'SwarmHC', 'OriginalHCO', 'OriginalHGS', 'OriginalHGSO', 'OriginalHHO', 'DevHS', 'OriginalHS', 'OriginalICA', 'OriginalINFO', 'OriginalIWO', 'DevJA', 'LevyJA', 'OriginalJA', 'DevLCO', 'ImprovedLCO', 'OriginalLCO', 'OriginalMA', 'OriginalMFO', 'OriginalMGO', 'OriginalMPA', 'OriginalMRFO', 'WMQIMRFO', 'OriginalMSA', 'DevMVO', 'OriginalMVO', 'OriginalNGO', 'ImprovedNMRA', 'OriginalNMRA', 'OriginalNRO', 'OriginalOOA', 'OriginalPFA', 'OriginalPOA', 'AIW_PSO', 'CL_PSO', 'C_PSO', 'HPSO_TVAC', 'LDW_PSO', 'OriginalPSO', 'P_PSO', 'OriginalPSS', 'DevQSA', 'ImprovedQSA', 'LevyQSA', 'OppoQSA', 'OriginalQSA', 'OriginalRIME', 'OriginalRUN', 'GaussianSA', 'OriginalSA', 'SwarmSA', 'DevSARO', 'OriginalSARO', 'DevSBO', 'OriginalSBO', 'DevSCA', 'OriginalSCA', 'QleSCA', 'OriginalSCSO', 'ImprovedSFO', 'OriginalSFO', 'L_SHADE', 'OriginalSHADE', 'OriginalSHIO', 'OriginalSHO', 'ImprovedSLO', 'ModifiedSLO', 'OriginalSLO', 'DevSMA', 'OriginalSMA', 'DevSOA', 'OriginalSOA', 'OriginalSOS', 'DevSPBO', 'OriginalSPBO', 'OriginalSRSR', 'DevSSA', 'OriginalSSA', 'OriginalSSDO', 'OriginalSSO', 'OriginalSSpiderA', 'OriginalSSpiderO', 'OriginalSTO', 'OriginalSeaHO', 'OriginalServalOA', 'OriginalTDO', 'DevTLO', 'ImprovedTLO', 'OriginalTLO', 'OriginalTOA', 'DevTPO', 'OriginalTS', 'OriginalTSA', 'OriginalTSO', 'EnhancedTWO', 'LevyTWO', 'OppoTWO', 'OriginalTWO', 'DevVCS', 'OriginalVCS', 'OriginalWCA', 'OriginalWDO', 'OriginalWHO', 'HI_WOA', 'OriginalWOA', 'OriginalWaOA', 'OriginalWarSO', 'OriginalZOA']

SUPPORTED_REG_OBJECTIVES = {'A10': 'max', 'A20': 'max', 'A30': 'max', 'ACOD': 'max', 'APCC': 'max', 'AR': 'max', 'AR2': 'max', 'CI': 'max', 'COD': 'max', 'COR': 'max', 'COV': 'max', 'CRM': 'min', 'DRV': 'min', 'EC': 'max', 'EVS': 'max', 'GINI': 'min', 'GINI_WIKI': 'min', 'JSD': 'min', 'KGE': 'max', 'MAAPE': 'min', 'MAE': 'min', 'MAPE': 'min', 'MASE': 'min', 'ME': 'min', 'MRB': 'min', 'MRE': 'min', 'MSE': 'min', 'MSLE': 'min', 'MedAE': 'min', 'NNSE': 'max', 'NRMSE': 'min', 'NSE': 'max', 'OI': 'max', 'PCC': 'max', 'PCD': 'max', 'R': 'max', 'R2': 'max', 'R2S': 'max', 'RAE': 'min', 'RMSE': 'min', 'RSE': 'min', 'RSQ': 'max', 'SMAPE': 'min', 'VAF': 'max', 'WI': 'max'}

build_model()

Builds the model architecture and sets the optimizer and loss function based on the task.

Raises:

ValueError – If the task is not recognized.

get_name()

Generate a descriptive name for the MLP model based on the optimizer.

Returns:

A string representing the name of the model, including details about the optimizer used. If self.optim is a string, the name will be formatted as “<self.optim_paras>-MLP”. Otherwise, it will return “<self.optimizer.name>-MLP”, assuming self.optimizer is an object with a name attribute.

Return type:

str

Notes

• This method relies on the presence of self.optim, self.optim_paras, and self.optimizer.name attributes within the model instance.

• It is intended to provide a consistent naming scheme for model instances based on the optimizer configuration.

objective_function(solution=None)

Evaluates the fitness function for classification metrics based on the provided solution.

Parameters:

solution (np.ndarray, default=None) – The proposed solution to evaluate.

Returns:

result – The fitness value, representing the loss for the current solution.

Return type:

float

set_optim_and_paras(optim=None, optim_paras=None)

Sets the optim and optim_paras parameters for this class.

Parameters:

• optim (str) – The optimizer name to be set.

• optim_paras (dict) – Parameters to configure the optimizer.

class metaperceptron.core.base_mlp.BaseMlp(hidden_layers, act_names, dropout_rates, task='classification', act_output=None, seed=None)

Bases: BaseEstimator

A custom MLP model compatible with sklearn, implemented using PyTorch. This class supports Multi-Layer Perceptron for both classification and regression tasks, with customizable hidden layers, activation functions, and dropout rates.

Parameters:

• hidden_layers (list of int, tuple of int, int) – Specifies the number of nodes in each hidden layer.

• act_names (list of str, tuple of str, str) – List of activation function names, one for each hidden layer.

• dropout_rates (list of float, tuple of float, float) – Dropout rates for each hidden layer (0 indicates no dropout).

• task (str, optional) – Task type, either “classification” or “regression”. Default is “classification”.

• act_output (str or None, optional) – Activation function for the output layer, default depends on the task type.

• seed (int or None, optional) – The seed value for the random number generator.

SUPPORTED_CLS_METRICS

Supported metrics for classification evaluation.

Type:

dict

SUPPORTED_REG_METRICS

Supported metrics for regression evaluation.

Type:

dict

SUPPORTED_CLS_METRICS = {'AS': 'max', 'BSL': 'min', 'CEL': 'min', 'CKS': 'max', 'F1S': 'max', 'F2S': 'max', 'FBS': 'max', 'GINI': 'min', 'GMS': 'max', 'HL': 'min', 'HS': 'max', 'JSI': 'max', 'KLDL': 'min', 'LS': 'max', 'MCC': 'max', 'NPV': 'max', 'PS': 'max', 'ROC-AUC': 'max', 'RS': 'max', 'SS': 'max'}

SUPPORTED_REG_METRICS = {'A10': 'max', 'A20': 'max', 'A30': 'max', 'ACOD': 'max', 'APCC': 'max', 'AR': 'max', 'AR2': 'max', 'CI': 'max', 'COD': 'max', 'COR': 'max', 'COV': 'max', 'CRM': 'min', 'DRV': 'min', 'EC': 'max', 'EVS': 'max', 'GINI': 'min', 'GINI_WIKI': 'min', 'JSD': 'min', 'KGE': 'max', 'MAAPE': 'min', 'MAE': 'min', 'MAPE': 'min', 'MASE': 'min', 'ME': 'min', 'MRB': 'min', 'MRE': 'min', 'MSE': 'min', 'MSLE': 'min', 'MedAE': 'min', 'NNSE': 'max', 'NRMSE': 'min', 'NSE': 'max', 'OI': 'max', 'PCC': 'max', 'PCD': 'max', 'R': 'max', 'R2': 'max', 'R2S': 'max', 'RAE': 'min', 'RMSE': 'min', 'RSE': 'min', 'RSQ': 'max', 'SMAPE': 'min', 'VAF': 'max', 'WI': 'max'}

evaluate(y_true, y_pred, list_metrics=None)

Evaluate the model using specified metrics.

Parameters:

• y_true (array-like) – True target values.

• y_pred (array-like) – Model’s predicted values.

• list_metrics (list of str, optional) – Names of metrics for evaluation (e.g., “MSE”, “MAE”).

Returns:

Evaluation metrics and their values.

Return type:

dict

fit(X, y)

Train the MLP model on the given dataset.

Parameters:

• X (array-like or torch.Tensor) – Training features.

• y (array-like or torch.Tensor) – Target values.

static load_model(load_path='history', filename='model.pkl') → EstimatorType

Load a model from a pickle file.

Parameters:

• load_path (str, optional) – Path to load the model from (default: “history”).

• filename (str, optional) – Filename of the saved model (default: “model.pkl”).

Returns:

The loaded model.

Return type:

BaseMlp

predict(X)

Generate predictions for input data using the trained model.

Parameters:

X (array-like or torch.Tensor) – Input features for prediction.

Returns:

Model predictions for each input sample.

Return type:

array-like or torch.Tensor

save_evaluation_metrics(y_true, y_pred, list_metrics=('RMSE', 'MAE'), save_path='history', filename='metrics.csv')

Save evaluation metrics to a CSV file.

Parameters:

• y_true (array-like) – Ground truth values.

• y_pred (array-like) – Model predictions.

• list_metrics (list of str, optional) – Metrics for evaluation (default: (“RMSE”, “MAE”)).

• save_path (str, optional) – Path to save the file (default: “history”).

• filename (str, optional) – Filename for saving metrics (default: “metrics.csv”).

save_model(save_path='history', filename='model.pkl')

Save the trained model to a pickle file.

Parameters:

• save_path (str, optional) – Path to save the model (default: “history”).

• filename (str, optional) – Filename for saving model, with “.pkl” extension (default: “model.pkl”).

save_training_loss(save_path='history', filename='loss.csv')

Save training loss history to a CSV file.

Parameters:

• save_path (str, optional) – Path to save the file (default: “history”).

• filename (str, optional) – Filename for saving loss history (default: “loss.csv”).

save_y_predicted(X, y_true, save_path='history', filename='y_predicted.csv')

Save true and predicted values to a CSV file.

Parameters:

• X (array-like or torch.Tensor) – Input features.

• y_true (array-like) – True values.

• save_path (str, optional) – Path to save the file (default: “history”).

• filename (str, optional) – Filename for saving predicted values (default: “y_predicted.csv”).

score(X, y)

Evaluate the model on the given dataset.

Parameters:

• X (array-like or torch.Tensor) – Evaluation features.

• y (array-like or torch.Tensor) – True values.

Returns:

The accuracy or evaluation score.

Return type:

float

set_seed(seed)

Set the random seed for the model to ensure reproducibility.

Parameters:

seed (int, None) – The seed value to use for random number generators within the model.

Notes

• This method stores the seed value in the self.seed attribute.

• Setting a seed helps achieve reproducible results, especially in training neural networks where randomness affects initialization and other stochastic operations.

class metaperceptron.core.base_mlp.BaseStandardMlp(hidden_layers=(100,), act_names='ReLU', dropout_rates=0.2, act_output=None, epochs=1000, batch_size=16, optim='Adam', optim_paras=None, early_stopping=True, n_patience=10, epsilon=0.001, valid_rate=0.1, seed=42, verbose=True)

Bases: BaseMlp

A custom standard MLP (Multi-Layer Perceptron) class that extends the BaseMlp class with additional features such as early stopping, validation, and various supported optimizers.

SUPPORTED_OPTIMIZERS

A list of optimizer names supported by the class.

Type:

list

Parameters:

• hidden_layers (tuple, list, int) – Number of neurons in each hidden layer.

• act_names (tuple, list, str) – Activation function(s) for each hidden layer.

• dropout_rates (tuple, list, float) – Dropout rate to prevent overfitting.

• act_output (str) – Activation function for the output layer.

• epochs (int) – Number of training epochs.

• batch_size (int) – Size of each training batch.

• optim (str) – Name of the optimizer to use from SUPPORTED_OPTIMIZERS.

• optim_paras (dict, optional) – Additional parameters for the optimizer.

• early_stopping (bool) – Flag to enable early stopping.

• n_patience (int) – Number of epochs to wait before stopping if no improvement.

• epsilon (float) – Minimum change to qualify as improvement.

• valid_rate (float) – Proportion of data to use for validation.

• seed (int) – Random seed for reproducibility.

• verbose (bool) – If True, outputs training progress.

SUPPORTED_OPTIMIZERS = ['Adafactor', 'Adadelta', 'Adagrad', 'Adam', 'Adamax', 'AdamW', 'ASGD', 'LBFGS', 'NAdam', 'RAdam', 'RMSprop', 'Rprop', 'SGD', 'SparseAdam']

build_model()

Build and initialize the MLP model, optimizer, and criterion based on user specifications.

This function sets up the model structure, optimizer type and parameters, and loss criterion depending on the task type (classification or regression).

process_data(X, y, **kwargs)

Process and prepare data for training.

Parameters:

• X (array-like) – Feature data for training.

• y (array-like) – Target labels or values for training.

• **kwargs (additional keyword arguments) – Additional parameters for data processing, if needed.

class metaperceptron.core.base_mlp.CustomMLP(size_input, size_output, hidden_layers, act_names, dropout_rates, task='classification', act_output=None, seed=None)

Bases: Module

A customizable multi-layer perceptron (MLP) model with flexible hidden layers, activations, and dropout rates, suitable for various tasks such as classification and regression.

- SUPPORTED_ACTIVATIONS

A list of supported activation function names.

Type:

list of str

- network

The constructed MLP network with layers.

Type:

nn.Sequential

Parameters:

• size_input (-) – Number of input features.

• size_output (-) – Number of output nodes.

• hidden_layers (-) – Number of nodes in each hidden layer.

• act_names (-) – Activation functions for each hidden layer.

• dropout_rates (-) – Dropout rates for each hidden layer.

• task (-) – Task type, “classification”, “binary_classification”, or “regression”.

• act_output (-) – Activation function for the output layer; uses default if None.

• seed (-) – The random seed for reproducibility

SUPPORTED_ACTIVATIONS = ['Threshold', 'ReLU', 'RReLU', 'Hardtanh', 'ReLU6', 'Sigmoid', 'Hardsigmoid', 'Tanh', 'SiLU', 'Mish', 'Hardswish', 'ELU', 'CELU', 'SELU', 'GLU', 'GELU', 'Hardshrink', 'LeakyReLU', 'LogSigmoid', 'Softplus', 'Softshrink', 'MultiheadAttention', 'PReLU', 'Softsign', 'Tanhshrink', 'Softmin', 'Softmax', 'Softmax2d', 'LogSoftmax']

forward(x)

Forward pass through the MLP model.

Parameters:

x (-) – The input tensor.

Returns:

The output of the MLP model.

Return type:

• torch.Tensor

get_weights()

Retrieve network weights as a flattened array.

Returns:

Flattened array of the model’s weights.

Return type:

• np.ndarray

get_weights_size()

Calculate the total number of trainable parameters in the model.

Returns:

Total number of parameters.

Return type:

• int

set_weights(solution)

Set network weights based on a given solution vector.

Parameters:

solution (-) – A flat array of weights to set in the model.

training: bool

class metaperceptron.core.base_mlp.EarlyStopper(patience=1, epsilon=0.01)

Bases: object

A utility class for implementing early stopping in training processes to prevent overfitting.

- patience

Number of consecutive epochs to tolerate no improvement before stopping.

Type:

int

- epsilon

Minimum loss improvement threshold to reset the patience counter.

Type:

float

- counter

Tracks the number of epochs without sufficient improvement.

Type:

int

- min_loss

Keeps track of the minimum observed loss.

Type:

float

early_stop(loss)

Checks if training should be stopped based on the current loss.

Parameters:

loss (-) – The current loss value for the epoch.

Returns:

True if training should stop, False otherwise.

Return type:

• bool

metaperceptron.core.gradient_mlp module

class metaperceptron.core.gradient_mlp.MlpClassifier(hidden_layers=(100,), act_names='ReLU', dropout_rates=0.2, act_output=None, epochs=1000, batch_size=16, optim='Adam', optim_paras=None, early_stopping=True, n_patience=10, epsilon=0.001, valid_rate=0.1, seed=42, verbose=True)

Bases: BaseStandardMlp, ClassifierMixin

Multi-layer Perceptron (MLP) Classifier that inherits from BaseStandardMlp and ClassifierMixin.

Parameters:

• hidden_layers (tuple, default=(100,)) – Defines the number of hidden layers and the units per layer in the network.

• act_names (str or list of str, default="ReLU") – Activation function(s) for each layer. Can be a single activation name for all layers or a list of names.

• dropout_rates (float or list of float, default=0.2) – Dropout rates for each hidden layer to prevent overfitting. If a single float, the same rate is applied to all layers.

• act_output (str, default=None) – Activation function for the output layer.

• epochs (int, default=1000) – Number of training epochs.

• batch_size (int, default=16) – Batch size used in training.

• optim (str, default="Adam") – Optimizer to use, selected from the supported optimizers.

• optim_paras (dict, default=None) – Parameters for the optimizer, such as learning rate, beta values, etc.

• early_stopping (bool, default=True) – If True, training will stop early if validation loss does not improve.

• n_patience (int, default=10) – Number of epochs to wait for an improvement in validation loss before stopping.

• epsilon (float, default=0.001) – Minimum improvement in validation loss to continue training.

• valid_rate (float, default=0.1) – Fraction of data to use for validation.

• seed (int, default=42) – Seed for random number generation.

• verbose (bool, default=True) – If True, prints training progress and validation loss during training.

evaluate(y_true, y_pred, list_metrics=('AS', 'RS'))

Returns performance metrics for the model on the provided test data.

Parameters:

• y_true (array-like of shape (n_samples,)) – True class labels.

• y_pred (array-like of shape (n_samples,)) – Predicted class labels.

• list_metrics (list, default=("AS", "RS")) – List of performance metrics to calculate. Refer to Permetrics (https://github.com/thieu1995/permetrics) library for available metrics.

Returns:

Dictionary with results for the specified metrics.

Return type:

dict

fit(X, y, **kwargs)

Trains the MLP model on the provided data.

Parameters:

• X (array-like, shape (n_samples, n_features)) – Training data.

• y (array-like, shape (n_samples,)) – Target values.

Returns:

self – Fitted classifier.

Return type:

object

predict(X)

Predicts the class labels for the given input data.

Parameters:

X (array-like, shape (n_samples, n_features)) – Input data.

Returns:

Predicted class labels for each sample.

Return type:

numpy.ndarray

predict_proba(X)

Computes the probability estimates for each class (for classification tasks only).

Parameters:

X (array-like, shape (n_samples, n_features)) – Input data.

Returns:

Probability predictions for each class.

Return type:

numpy.ndarray

process_data(X, y, **kwargs)

Prepares and processes data for training, including optional splitting into validation data.

Parameters:

• X (array-like, shape (n_samples, n_features)) – Training data.

• y (array-like, shape (n_samples,)) – Target values.

Returns:

tuple – Data loader for training data, and tensors for validation data (if specified).

Return type:

(train_loader, X_valid_tensor, y_valid_tensor)

score(X, y)

Computes the accuracy score for the classifier.

Parameters:

• X (array-like, shape (n_samples, n_features)) – Input data.

• y (array-like, shape (n_samples,)) – True class labels.

Returns:

Accuracy score of the classifier.

Return type:

float

class metaperceptron.core.gradient_mlp.MlpRegressor(hidden_layers=(100,), act_names='ELU', dropout_rates=0.2, act_output=None, epochs=1000, batch_size=16, optim='Adam', optim_paras=None, early_stopping=True, n_patience=10, epsilon=0.001, valid_rate=0.1, seed=42, verbose=True)

Bases: BaseStandardMlp, RegressorMixin

Multi-Layer Perceptron (MLP) Regressor for predicting continuous values.

Parameters:

• hidden_layers (tuple, optional) – A tuple indicating the number of neurons in each hidden layer (default is (100,)).

• act_names (str or list of str, optional) – Activation function(s) for hidden layers (default is “ELU”).

• dropout_rates (float or list of float, optional) – Dropout rate(s) for regularization (default is 0.2).

• act_output (str, optional) – Activation function for the output layer (default is None).

• epochs (int, optional) – Number of epochs for training (default is 1000).

• batch_size (int, optional) – Size of the mini-batch during training (default is 16).

• optim (str, optional) – Optimization algorithm (default is “Adam”).

• optim_paras (dict, optional) – Additional parameters for the optimizer (default is None).

• early_stopping (bool, optional) – Flag to enable early stopping during training (default is True).

• n_patience (int, optional) – Number of epochs to wait for improvement before stopping (default is 10).

• epsilon (float, optional) – Tolerance for improvement (default is 0.001).

• valid_rate (float, optional) – Proportion of data to use for validation (default is 0.1).

• seed (int, optional) – Random seed for reproducibility (default is 42).

• verbose (bool, optional) – Flag to enable verbose output during training (default is True).

evaluate(y_true, y_pred, list_metrics=('MSE', 'MAE'))

Returns a list of performance metrics for the predictions.

Parameters:

• y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for the input features.

• y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Predicted values for the input features.

• list_metrics (list, default=("MSE", "MAE")) – List of metrics to evaluate (can be from Permetrics library: https://github.com/thieu1995/permetrics).

Returns:

results – A dictionary containing the results of the specified metrics.

Return type:

dict

fit(X, y, **kwargs)

Fits the MLP model to the provided training data.

Parameters:

• X (array-like) – Input features for training.

• y (array-like) – Target values for training.

Returns:

self – Returns the instance of the fitted model.

Return type:

MlpRegressor

predict(X)

Predicts the target values for the given input features.

Parameters:

X (array-like) – Input features for prediction.

Returns:

Predicted values for the input features.

Return type:

numpy.ndarray

process_data(X, y, **kwargs)

Prepares the input data for training and validation by converting it to tensors and creating DataLoaders for batch processing.

Parameters:

• X (array-like) – Input features for the regression task.

• y (array-like) – Target values for the regression task.

Returns:

A tuple containing the training DataLoader and optional validation tensors.

Return type:

tuple

score(X, y)

Computes the R2 score of the predictions.

Parameters:

• X (array-like) – Input features for scoring.

• y (array-like) – True target values for the input features.

Returns:

R2 score indicating the model’s performance.

Return type:

float

metaperceptron.core.metaheuristic_mlp module

class metaperceptron.core.metaheuristic_mlp.MhaMlpClassifier(hidden_layers=(100,), act_names='ReLU', dropout_rates=0.2, act_output=None, optim='BaseGA', optim_paras=None, obj_name='F1S', seed=42, verbose=True)

Bases: BaseMhaMlp, ClassifierMixin

A Metaheuristic-based MLP Classifier that extends the BaseMhaMlp class and implements the ClassifierMixin interface from Scikit-Learn for classification tasks.

classes_

Unique classes found in the target variable.

Type:

np.ndarray

metric_class

The metric class used for evaluating classification performance.

Type:

type

Parameters:

• hidden_layers (list of int, tuple of int, or np.ndarray of int) – The structure of the hidden layers (default is (100,)).

• act_names (str) – Activation function name to use in hidden layers (default is “ReLU”).

• dropout_rates (float, optional) – Dropout rate for regularization (default is 0.2).

• act_output (any, optional) – Activation function for the output layer (default is None).

• optim (str) – The optimization algorithm to use (default is “BaseGA”).

• optim_paras (dict) – Parameters for the optimizer (default is None).

• obj_name (str, optional) – The objective name for the optimization (default is “F1S”).

• seed (int, optional) – Random seed for reproducibility (default is 42).

• verbose (bool, optional) – Whether to print detailed logs during fitting (default is True).

evaluate(y_true, y_pred, list_metrics=('AS', 'RS'))

Return the list of performance metrics on the given test data and labels.

Parameters:

• y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Predicted values for X.

• list_metrics (list, default=("AS", "RS")) – List of metrics to compute using Permetrics library: https://github.com/thieu1995/permetrics

Returns:

results – A dictionary containing the results of the requested metrics.

Return type:

dict

fit(X, y, lb=(-1.0,), ub=(1.0,), mode='single', n_workers=None, termination=None, save_population=False, **kwargs)

Fits the model to the training data.

Parameters:

• X (array-like, shape (n_samples, n_features)) – Training data.

• y (array-like, shape (n_samples,)) – Target values.

• lb (tuple, optional) – Lower bounds for optimization (default is (-1.0,)).

• ub (tuple, optional) – Upper bounds for optimization (default is (1.0,)).

• mode (str, optional) – Mode for optimization (default is ‘single’).

• n_workers (int, optional) – Number of workers for parallel processing (default is None).

• termination (any, optional) – Termination criteria for optimization (default is None).

• save_population (bool, optional) – Whether to save the population during optimization (default is False).

• **kwargs (additional parameters) – Additional parameters for fitting.

Returns:

self – Returns the instance of the fitted model.

Return type:

MhaMlpClassifier

predict(X)

Predicts the class labels for the provided input data.

Parameters:

X (array-like, shape (n_samples, n_features)) – Input data for prediction.

Returns:

Predicted class labels for each sample.

Return type:

np.ndarray

predict_proba(X)

Computes the probability estimates for each class (for classification tasks only).

Parameters:

X (array-like, shape (n_samples, n_features)) – Input data for which to predict probabilities.

Returns:

Probability predictions for each class.

Return type:

np.ndarray

Raises:

ValueError – If the task is not a classification task.

score(X, y)

Computes the accuracy score of the model based on predictions.

Parameters:

• X (array-like, shape (n_samples, n_features)) – Input data for scoring.

• y (array-like, shape (n_samples,)) – True labels for comparison.

Returns:

Accuracy score of the model.

Return type:

float

set_fit_request(*, lb: bool | None | str = '$UNCHANGED$', mode: bool | None | str = '$UNCHANGED$', n_workers: bool | None | str = '$UNCHANGED$', save_population: bool | None | str = '$UNCHANGED$', termination: bool | None | str = '$UNCHANGED$', ub: bool | None | str = '$UNCHANGED$') → MhaMlpClassifier

Request metadata passed to the fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to fit.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

• lb (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for lb parameter in fit.

• mode (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for mode parameter in fit.

• n_workers (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for n_workers parameter in fit.

• save_population (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for save_population parameter in fit.

• termination (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for termination parameter in fit.

• ub (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for ub parameter in fit.

Returns:

self – The updated object.

Return type:

object

class metaperceptron.core.metaheuristic_mlp.MhaMlpRegressor(hidden_layers=(100,), act_names='ELU', dropout_rates=0.2, act_output=None, optim='BaseGA', optim_paras=None, obj_name='MSE', seed=42, verbose=True)

Bases: BaseMhaMlp, RegressorMixin

A Metaheuristic-based MLP Regressor that extends the BaseMhaMlp class and implements the RegressorMixin interface from Scikit-Learn for regression tasks.

metric_class

The metric class used for evaluating regression performance.

Type:

type

Parameters:

• hidden_layers (list of int, tuple of int, or np.ndarray of int) – The structure of the hidden layers (default is (100,)).

• act_names (str) – Activation function name to use in hidden layers (default is “ELU”).

• dropout_rates (float, optional) – Dropout rate for regularization (default is 0.2).

• act_output (any, optional) – Activation function for the output layer (default is None).

• optim (str) – The optimization algorithm to use (default is “BaseGA”).

• optim_paras (dict) – Parameters for the optimizer (default is None).

• obj_name (str, optional) – The objective name for the optimization (default is “MSE”).

• seed (int, optional) – Random seed for reproducibility (default is 42).

• verbose (bool, optional) – Whether to print detailed logs during fitting (default is True).

evaluate(y_true, y_pred, list_metrics=('AS', 'RS'))

Return the list of performance metrics on the given test data and labels.

Parameters:

• y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True values for X.

• y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) – Predicted values for X.

• list_metrics (list, default=("AS", "RS")) – List of metrics to compute using Permetrics library: https://github.com/thieu1995/permetrics

Returns:

results – A dictionary containing the results of the requested metrics.

Return type:

dict

fit(X, y, lb=(-1.0,), ub=(1.0,), mode='single', n_workers=None, termination=None, save_population=False, **kwargs)

Fits the model to the training data.

Parameters:

• X (array-like, shape (n_samples, n_features)) – Training data.

• y (array-like, shape (n_samples,) or (n_samples, n_outputs)) – Target values.

• lb (tuple, optional) – Lower bounds for optimization (default is (-1.0,)).

• ub (tuple, optional) – Upper bounds for optimization (default is (1.0,)).

• mode (str, optional) – Mode for optimization (default is ‘single’).

• n_workers (int, optional) – Number of workers for parallel processing (default is None).

• termination (any, optional) – Termination criteria for optimization (default is None).

• save_population (bool, optional) – Whether to save the population during optimization (default is False).

• **kwargs (additional parameters) – Additional parameters for fitting.

Returns:

self – Returns the instance of the fitted model.

Return type:

MhaMlpRegressor

predict(X)

Predicts the output values for the provided input data.

Parameters:

X (array-like, shape (n_samples, n_features)) – Input data for prediction.

Returns:

Predicted output values for each sample.

Return type:

np.ndarray

score(X, y)

Computes the R2 score of the model based on predictions.

Parameters:

• X (array-like, shape (n_samples, n_features)) – Input data for scoring.

• y (array-like, shape (n_samples,)) – True labels for comparison.

Returns:

R2 score of the model.

Return type:

float

set_fit_request(*, lb: bool | None | str = '$UNCHANGED$', mode: bool | None | str = '$UNCHANGED$', n_workers: bool | None | str = '$UNCHANGED$', save_population: bool | None | str = '$UNCHANGED$', termination: bool | None | str = '$UNCHANGED$', ub: bool | None | str = '$UNCHANGED$') → MhaMlpRegressor

Request metadata passed to the fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to fit.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

• lb (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for lb parameter in fit.

• mode (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for mode parameter in fit.

• n_workers (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for n_workers parameter in fit.

• save_population (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for save_population parameter in fit.

• termination (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for termination parameter in fit.

• ub (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for ub parameter in fit.

Returns:

self – The updated object.

Return type:

object

metaperceptron.core.comparator module

class metaperceptron.core.comparator.MhaMlpComparator(optim_dict=None, task='classification', hidden_layers=(10,), act_names='ELU', dropout_rates=None, act_output=None, obj_name=None, verbose=False, seed=None, **kwargs)

Bases: object

Automated compare different MhaMlp models based on provided optimizer configurations.

This class facilitates the comparison of multiple MhaMlp models with varying optimizer configurations. It provides methods for cross-validation and train-test split evaluation.

Parameters:

• optim_dict (dict, optional) – A dictionary of optimizer names and parameters.

• task (str, optional) – The task type (classification or regression). Defaults to ‘classification’.

• hidden_layers (int, list, tuple, optional) – The number of nodes in each hidden layer. Defaults is (10, ).

• act_names (str, optional) – The activation function name. Defaults to ‘ELU’.

• obj_name (str, optional) – The objective function name. Defaults to None.

• verbose (bool, optional) – Whether to print verbose output. Defaults to False.

• seed (int, optional) – Random seed for reproducibility. Defaults to None.

• obj_weights (array-like, optional) – Weights for the objective function. Defaults to None.

• **kwargs – Additional keyword arguments for model initialization.

compare_cross_val_score(X, y, metric=None, cv=5, n_trials=10, to_csv=True, saved_file_path='history/results_cross_val_score.csv', **kwargs)

Performs cross-validation with a single metric.

Compares different MhaMlp models using cross-validation with a single metric.

Parameters:

• X (array-like) – The feature matrix.

• y (array-like) – The target vector.

• metric (str, optional) – The metric to evaluate. Defaults to None.

• cv (int, optional) – The number of cross-validation folds. Defaults to 5.

• n_trials (int, optional) – The number of trials. Defaults to 10.

• to_csv (bool, optional) – Whether to save results to a CSV file. Defaults to True.

• saved_file_path (str, optional) – The path to save the CSV file. Defaults to ‘history/results_cross_val_score.csv’.

• **kwargs – Additional keyword arguments for cross_val_score.

Returns:

The comparison results.

Return type:

pandas.DataFrame

compare_cross_validate(X, y, metrics=None, cv=5, return_train_score=True, n_trials=10, to_csv=True, saved_file_path='history/results_cross_validate.csv', **kwargs)

Performs cross-validation for model comparison.

Compares different MhaMlp models using cross-validation.

Parameters:

• X (array-like) – The feature matrix.

• y (array-like) – The target vector.

• metrics (list, optional) – A list of metric names. Defaults to None.

• cv (int, optional) – The number of cross-validation folds. Defaults to 5.

• return_train_score (bool, optional) – Whether to return train scores. Defaults to True.

• n_trials (int, optional) – The number of trials. Defaults to 10.

• to_csv (bool, optional) – Whether to save results to a CSV file. Defaults to True.

• saved_file_path (str, optional) – The path to save the CSV file. Defaults to ‘history/results_cross_validate.csv’.

• **kwargs – Additional keyword arguments for cross_validate.

Returns:

The comparison results.

Return type:

pandas.DataFrame

compare_train_test(X_train, y_train, X_test, y_test, metrics=None, n_trials=10, to_csv=True, saved_file_path='history/results_train_test.csv')

Compares models using train-test split.

Compares different MhaMlp models using train-test split evaluation.

Parameters:

• X_train (array-like) – The training feature matrix.

• y_train (array-like) – The training target vector.

• X_test (array-like) – The testing feature matrix.

• y_test (array-like) – The testing target vector.

• metrics (list, optional) – A list of metric names. Defaults to None.

• n_trials (int, optional) – The number of trials. Defaults to 10.

• to_csv (bool, optional) – Whether to save results to a CSV file. Defaults to True.

• saved_file_path (str, optional) – The path to save the CSV file. Defaults to ‘history/results_train_test.csv’.

Returns:

The comparison results.

Return type:

pandas.DataFrame

metaperceptron.core.tuner module

class metaperceptron.core.tuner.MhaMlpTuner(task='classification', param_dict=None, search_method='gridsearch', scoring=None, cv=3, **kwargs)

Bases: object

Automated hyperparameter tuner for MhaMlp models.

Performs hyperparameter tuning for MhaMlp models using either GridSearchCV or RandomizedSearchCV. Provides an interface for fitting and predicting using the best found model.

model_class

The MhaMlp model class (MhaMlpRegressor or MhaMlpClassifier).

Type:

class

param_grid

The parameter grid for hyperparameter tuning.

Type:

dict

search_method

The optimization method (‘gridsearch’ or ‘randomsearch’).

Type:

str

kwargs

Additional keyword arguments for the search method.

Type:

dict

searcher

The searcher

Type:

GridSearchCV or RandomizedSearchCV

best_estimator_

The best estimator found during tuning.

Type:

sklearn.base.BaseEstimator

best_params_

The best hyperparameters found during tuning.

Type:

dict

fit(X, y)

Fits the tuner to the data and tunes hyperparameters.

predict(X)

Predicts using the best estimator.

fit(X, y)

Fits the tuner to the data and tunes the hyperparameters.

Parameters:

• X (array-like) – Training features.

• y (array-like) – Training target values.

Returns:

Fitted tuner object.

Return type:

self

predict(X)

Predicts using the best estimator found during tuning.

Parameters:

X (array-like) – Data to predict.

Returns:

Predicted values.

Return type:

array-like