graphslim.evaluation package

graphslim.evaluation.eval_agent module

class graphslim.evaluation.eval_agent.Evaluator(args, **kwargs)[source]

Bases: object

A class to evaluate different models and their hyperparameters on graph data.

Parameters:

  • args (argparse.Namespace) – Command-line arguments and configuration parameters.

  • **kwargs (keyword arguments) – Additional parameters.

MIA_evaluate(data, model_type, verbose=True, reduced=True, mode='eval')[source]

Evaluates a model over multiple runs and returns mean and standard deviation of accuracy.

Parameters:

  • data (Dataset) – The dataset containing the graph data.

  • model_type (str) – The type of model to evaluate.

  • verbose (bool, optional, default=True) – Whether to print detailed logs.

  • reduced (bool, optional, default=True) – Whether to use synthetic data.

  • mode (str, optional, default='eval') – The mode for the model (e.g., ‘eval’ or ‘cross’).

Returns:

  • mean_acc (float) – Mean accuracy over multiple runs.

  • std_acc (float) – Standard deviation of accuracy over multiple runs.

evaluate(data, model_type, verbose=True, reduced=True, mode='eval')[source]

Evaluates a model over multiple runs and returns mean and standard deviation of accuracy.

Parameters:

  • data (Dataset) – The dataset containing the graph data.

  • model_type (str) – The type of model to evaluate.

  • verbose (bool, optional, default=True) – Whether to print detailed logs.

  • reduced (bool, optional, default=True) – Whether to use synthetic data.

  • mode (str, optional, default='eval') – The mode for the model (e.g., ‘eval’ or ‘cross’).

Returns:

  • mean_acc (float) – Mean accuracy over multiple runs.

  • std_acc (float) – Standard deviation of accuracy over multiple runs.

Performs a grid search over hyperparameters.

Parameters:

  • data (Dataset) – The dataset containing the graph data.

  • model_type (str) – The type of model used for evaluation.

  • param_grid (dict) – A dictionary containing parameter grids for grid search.

  • reduced (bool, optional, default=True) – Whether to use synthetic data.

Returns:

  • best_test_result (tuple) – Best test result as (mean_accuracy, std_accuracy).

  • best_params (dict) – Best parameters found during grid search.

nas_evaluate(data, model_type, verbose=False, reduced=None)[source]

Evaluates a model for neural architecture search (NAS) and returns mean and standard deviation of validation accuracy.

Parameters:

  • data (Dataset) – The dataset containing the graph data.

  • model_type (str) – The type of model to evaluate.

  • verbose (bool, optional, default=False) – Whether to print detailed logs.

  • reduced (bool, optional, default=None) – Whether to use synthetic data.

Returns:

  • mean_acc_val (float) – Mean validation accuracy over multiple runs.

  • std_acc_val (float) – Standard deviation of validation accuracy over multiple runs.

reset_parameters()[source]

Initializes or resets model parameters.

test(data, model_type, verbose=True, reduced=True, mode='eval', MIA=False)[source]

Tests a model and returns accuracy and loss.

Parameters:

  • data (Dataset) – The dataset containing the graph data.

  • model_type (str) – The type of model to test.

  • verbose (bool, optional, default=True) – Whether to print detailed logs.

  • reduced (bool, optional, default=True) – Whether to use synthetic data.

  • mode (str, optional, default='eval') – The mode for the model (e.g., ‘eval’ or ‘cross’).

Returns:

  • best_acc_val (float) – Best accuracy on validation set.

  • acc_test (float) – Accuracy on test set.

train_cross(data, grid_search=True, reduced=True)[source]

Trains models and performs grid search if required.

Parameters:

  • data (Dataset) – The dataset containing the graph data.

  • grid_search (bool, optional, default=True) – Whether to perform grid search over hyperparameters.

  • reduced (bool, optional, default=True) – Whether to use synthetic data.

tsne_vis(feat_train, labels_train, feat_syn, labels_syn)[source]

Visualize t-SNE for original and synthetic data.

Parameters:

  • feat_train (torch.tensor) – Original features.

  • labels_train (torch.tensor) – Labels for original features.

  • feat_syn (torch.tensor) – Synthetic features.

  • labels_syn (torch.tensor) – Labels for synthetic features.

visualize(data)[source]

Visualizes synthetic and original data using t-SNE and saves the plot as a PDF file.

Parameters:

  • args (argparse.Namespace) – Command-line arguments and configuration parameters.

  • data (Dataset) – The dataset containing the graph data.

graphslim.evaluation.nas_eval module

class graphslim.evaluation.nas_eval.NasEvaluator(args)[source]

Bases: object

Class for evaluating neural architecture search (NAS) performance on original and synthetic graphs.

cal_pearson()[source]

Calculates Pearson correlation coefficients between synthetic and original results.

Returns:

  • pearson_corr_acc (float) – Pearson correlation coefficient of accuracies.

  • pearson_corr_rank (float) – Pearson correlation coefficient of ranks.

evaluate_ori(data)[source]

Evaluates various architectures on the original graph and identifies the best one.

Parameters:

data (Dataset) – The dataset containing the graph data.

evaluate_syn(data)[source]

Evaluates various architectures on the synthetic graph and identifies the best one.

Parameters:

data (Dataset) – The dataset containing the graph data.

get_rank(results)[source]

Ranks results based on their values.

Parameters:

results (list of float) – The list of results to rank.

Returns:

ranks – The list of ranks corresponding to the results.

Return type:

list of int

test_params_ori(data)[source]

Tests the best architecture on the original graph using the best parameters.

Parameters:

data (Dataset) – The dataset containing the graph data.

test_params_syn(data)[source]

Tests the best architecture on the synthetic graph using the best parameters.

Parameters:

data (Dataset) – The dataset containing the graph data.

Returns:

acc_test_syn – The test accuracy on the synthetic graph.

Return type:

float

graphslim.evaluation.utils module

graphslim.evaluation.utils.calc_f1(y_true, y_pred, is_sigmoid=False)[source]

Calculate the F1 score for binary or multi-class classification.

This function calculates both the micro-averaged and macro-averaged F1 scores. The y_pred values are processed differently based on whether the classification uses sigmoid activation or not.

Parameters:

  • y_true (array-like, shape (n_samples,)) – True labels or ground truth values.

  • y_pred (array-like, shape (n_samples,) or (n_samples, n_classes)) – Predicted labels or probabilities. If is_sigmoid is True, this should be probabilities. Otherwise, it should be class predictions.

  • is_sigmoid (bool) – Flag indicating whether the classification uses sigmoid activation (binary classification) or not (multi-class classification). If True, y_pred contains probabilities; if False, y_pred contains class predictions.

Returns:

  • micro-averaged F1 score.

  • macro-averaged F1 score.

Return type:

tuple of float

graphslim.evaluation.utils.calculate_homophily(y, adj)[source]
graphslim.evaluation.utils.evaluate(output, labels, args)[source]

Evaluate the model performance based on the output and labels.

This function computes performance metrics depending on the type of dataset. For certain datasets, it calculates F1 scores. For others, it computes loss and accuracy.

Parameters:

  • output (torch.Tensor) – The model’s output logits or probabilities.

  • labels (torch.Tensor) – The ground truth labels.

  • args (Namespace) – Arguments that include dataset information to determine which metrics to use.

Return type:

None

graphslim.evaluation.utils.getsize_mb(elements)[source]

Calculate the total size of a list of elements in megabytes.

Parameters:

elements (list) – List of elements to calculate the size for. The elements can be SparseTensor, csr_matrix, or tensors.

Returns:

size – Total size of all elements in the list in megabytes.

Return type:

float

Examples

>>> elements = [tensor1, sparse_tensor, csr_matrix]
>>> getsize_mb(elements)
12.34
graphslim.evaluation.utils.inference_via_confidence(confidence_mtx1, confidence_mtx2, label_vec1, label_vec2)[source]
graphslim.evaluation.utils.verbose_time_memory(func)[source]

A decorator that measures and prints the execution time and memory usage of the decorated function.

This decorator prints the time taken by the function to execute in both seconds and milliseconds, and the memory usage of the data before and after the function call if verbose mode is enabled.

Parameters:

func (callable) – The function to be decorated.

Returns:

The wrapped function with added timing and memory usage functionality.

Return type:

callable