the_utils package

Subpackages

the_utils.evaluation package

Submodules

the_utils.bot module

Ding Message Utils

the_utils.bot.get_card(title, text, btn_orientation, btns)[source]

the_utils.bot.send(action_card, token, msgtype='actionCard')[source]

the_utils.bot.notify(token: str = 'bb68fb0c27bef0f856b72b6301d024d5fa1aaacba2d6963d27d267c673dbdf8e', text: str = 'This is a test message', title: str = 'Come from the_utils', btn_orientation: str = '0', btns: List | None = None)[source]

Send messages to DingTalk.

Parameters:

text (str, optional) – Message text, supports markdown syntax. Defaults to “This is a test message”.
title (str, optional) – Message title. Defaults to “Come from the_utils”.
btn_orientation (str, optional) – Interactive button location. Defaults to ‘0’.
btns (List, optional) – Interactive button settings. Defaults to None.

the_utils.common module

Common Utils

the_utils.common.format_result(dataset: str, model: str, source: str | None = None, sort_kw: bool = False, timezone='Asia/Shanghai', **kwargs) → Dict[source]

format results.

Parameters:

dataset (str) – dataset name.
model (str) – model name.
source (str, optional) – data source. Defaults to None.
sort_kw (bool, optional) – sort the kwargs with lowercase. Defaults to False.
timezone (str, optional) – timezone. Defaults to “Asia/Shanghai”.

Returns:

results dict.

Return type:

Dict

the_utils.common.get_str_time(timezone='Asia/Shanghai')[source]: Return localtime in the format of %Y-%m-%d-%H:%M:%S.

the_utils.common.format_value(value) → Any[source]

Return number as string with comma split.

Parameters:: value (int) – number.
Returns:: string of the number with comma split.
Return type:: str

the_utils.common.tab_printer(args: Dict, thead: List[str] | None = None, cols_align: List[str] | None = None, cols_valign: List[str] | None = None, cols_dtype: List[str] | None = None, sort: bool = True, verbose: bool | int = True) → None[source]

Function to print the logs in a nice tabular format.

Parameters:

args (Dict) – value dict.
thead (List[str], optional) – table head. Defaults to None.
cols_align (List[str], optional) – horizontal alignment of the columns. Defaults to None.
cols_valign (List[str], optional) – vertical alignment of the columns. Defaults to None.
cols_dtype (List[str], optional) – value types of the columns. Defaults to None.
sort (bool, optional) – whether to sort the keys. Defaults to True.

Returns:

table string to print.

Return type:

str

the_utils.file module

File Management.

the_utils.file.make_parent_dirs(target_path: PurePath) → None[source]

make all the parent dirs of the target path.

Parameters:: target_path (PurePath) – target path.

the_utils.file.refresh_file(target_path: str | None = None) → None[source]

clear target path

Parameters:: target_path (str) – file path

the_utils.file.csv2file(target_path: str, thead: List[str] | None = None, tbody: List[Any] | None = None, refresh: bool = False, is_dict_list: bool = False, sort_head: bool = False) → None[source]

save data to target_path of a csv file.

Parameters:

target_path (str) – target path
thead (List[str], optional) – csv table header, only written into the file when it is not None and file is empty. Defaults to None.
tbody (List, optional) – csv table content. Defaults to None.
refresh (bool, optional) – whether to clean the file first. Defaults to False.
is_dict_list (bool, optional) – whether the tbody is in the format of a list of dicts. Defaults to False.
sort_head (bool, optional) – whether to sort the head with lowercase before writing. Defaults to False.

Example

from the_utils import csv2file
save_file = "./results/example.csv"
final_params = {
    "dataset": "cora",
    "acc": "99.1",
    "NMI": "89.0",
}
thead=[]
# list of values
csv2file(
    target_path=save_file,
    thead=list(final_params.keys()),
    tbody=list(final_params.values()),
    refresh=False,
    is_dict_list=False,
)
# list of dicts
csv2file(
    target_path=save_file,
    tbody=[
        {
            "a": 1,
            "b": 2
        },
        {
            "a": 2,
            "b": 1
        },
    ],
    is_dict_list=True,
)

the_utils.file.save_to_csv_files(results: dict, add_info: dict, csv_name: str, save_path='./results', sort_head: bool = False) → None[source]

Save the evaluation results to a local csv file.

Parameters:

results (dict) – Evaluation results document.
add_info (dict) – Additional information, such as data set name, method name.
csv_name (str) – csv file name to store.
save_path (str, optional) – Folder path to store. Defaults to ‘./results’.
sort_head (bool, optional) – whether to sort the head before writing. Defaults to False.

Example

from the_utils import evaluate_from_embed_file
from the_utils import save_to_csv_files

method_name='orderedgnn'
data_name='texas'

clustering_res, classification_res = evaluate_from_embed_file(
    f'{data_name}_{method_name}_embeds.pth',
    f'{data_name}_data.pth',
    save_path='./save/',
)

add_info = {'data': data_name, 'method': method_name,}
save_to_csv_files(clustering_res, add_info, 'clutering.csv')
save_to_csv_files(classification_res, add_info, 'classification.csv')

the_utils.plt module

Draw plots.

the_utils.plt.charts(t, ax: <module 'matplotlib.axes' from '/graph/guming/projects/the_utils/.env/lib/python3.8/site-packages/matplotlib/axes/__init__.py'>)[source]

the_utils.plt.draw_chart(ys: ~typing.List[~typing.List], lbls: ~typing.List[str], y_colors: ~typing.List[str] = None, xs: ~typing.List[~typing.Any] = None, fill_between: ~typing.List = None, fill_colors: ~typing.List[str] = None, fill_alpha: ~typing.List[float] = 0.5, figsize: ~typing.Tuple = (24, 3), xmode: str = 's', ymode: str = 's', title: str = None, set_xticks: bool = False, xticks: ~typing.List[~typing.Any] = None, tick_labels: ~typing.List[~typing.Any] = None, x_rotation: float = 45, boxes: ~typing.List[~typing.List[int]] = None, box_colors: ~typing.List[str] = None, box_alphas: ~typing.List[float] = None, ylim: (<class 'float'>, <class 'float'>) = None, xlim: (<class 'float'>, <class 'float'>) = None, linewidth: ~typing.List[int] = None, types: ~typing.List[str] = 'line', markersize: float = 1.25, save_path: str = None, bar_width: float = 0.25, legend_loc: str = 'center left', legend_bbox_to_anchor: ~typing.Tuple[float] = (1, 0.5)) → None[source]

Draw charts.

Parameters:

ys (List[List]) – value lists of the y axis.
lbls (List[str]) – legend labels of the value lists.
y_colors (List[str], optional) – colors of the the ys. Defaults to None.
xs (List[Any], optional) – value list of the x axis. If None, idx will be used. Defaults to None.
fill_between (List, optional) – fill the range of [ys[idx][i] - fill_between[idx][i], ys[idx][i] + fill_between[idx][i]]. Defaults to None.
fill_colors (List[str], optional) – colors of the fill range. Defaults to None.
fill_alpha (List[float] or float, optional) – color alphas of the fill range. Defaults to 0.5.
figsize (Tuple, optional) – size of the output figure. Defaults to (24, 3).
xmode (str, optional) – all ys with the same x (‘s’) or with different xs (‘d’). Defaults to “s”.
ymode (str, optional) – all ys with the same tick (‘s’) or with different ticks (‘d’). Defaults to “s”.
title (str, optional) – figure title. Defaults to None.
set_xticks (bool, optional) – set the x ticks. Defaults to True.
xticks (List[Any], optional) – ticks for the x axis. Defaults to None.
tick_labels (List[Any], optional) – tick labels for the x axis. Defaults to None.
x_rotation (float, optional) – rotation of the x ticks. Defaults to 45.
boxes (List[List[int]], optional) – draw boxes. Defaults to None.
box_colors (List[str], optional) – colors of the boxes to draw. Defaults to None.
box_alpha (List[float], optional) – color alphas of the boxes to draw. Defaults to 0.5.
ylim (float, optional) – maximum value of the y axis. Defaults to None.
xlim (float, optional) – maximum value of the x axis. Defaults to None.
linewidth (List[int], optional) – line width of the line charts to draw. Defaults to None.
types (List[str] or str, optional) – types of charts of the ys. Defaults to “line”.
markersize (float, optional) – dot size of the scatter charts. Defaults to 1.25.
save_path (str, optional) – If not None, save image to the path. Defaults to None.
bar_width (float, optional) – width of the bars. Defaults to 0.25.
legend_loc (str, optional) – location of the legend. Defaults to “center left”.
legend_bbox_to_anchor (Tuple[float], optional) – bbox_to_anchor of the legend. Defaults to (1, 0.5).

Raises:

ValueError – “The arg ‘types:’{types} has values not supported.”

the_utils.save_load module

Tmp saving and loading.

the_utils.save_load.load_dict(filename_)[source]

the_utils.save_load.save_dict(di_, filename_)[source]

the_utils.save_load.get_modelfile_path(model_filename: str) → PurePath[source]

the_utils.save_load.check_modelfile_exists(model_filename: str) → bool[source]

the_utils.save_load.save_model(model_filename: str, model: Module, optimizer: Optimizer, current_epoch: int, loss: float) → None[source]

Save model, optimizer, current_epoch, loss to .checkpoints/${model_filename}.pt.

Parameters:

model_filename (str) – filename to save model.
model (torch.nn.Module) – model.
optimizer (torch.optim.Optimizer) – optimizer.
current_epoch (int) – current epoch.
loss (float) – loss.

the_utils.save_load.load_model(model_filename: str, model: Module, optimizer: Optimizer | None = None, device: device = device(type='cpu')) → Tuple[Module, Optimizer, int, float][source]

Load model from .checkpoints/${model_filename}.pt.

Parameters:

model_filename (str) – filename to load model.
model (torch.nn.Module) – model.
optimizer (torch.optim.Optimizer) – optimizer.

Returns:

[model, optimizer, epoch, loss]

Return type:

Tuple[torch.nn.Module, torch.optim.Optimizer, int, float]

the_utils.save_load.save_embedding(node_embeddings: tensor, dataset_name: str, model_name: str, params: dict, save_dir: str = './save', verbose: bool | int = True)[source]

Save embeddings.

Parameters:

node_embeddings (torch.tensor) – node embeddings.
dataset_name (str) – dataset name.
model_name (str) – model name.
params (dict) – parameter dict.
save_dir (str, optional) – save dir. Defaults to “./save”.
verbose (Union[bool, int], optional) – whether to print debug info. Defaults to True.

the_utils.setting module

Utils for Settings.

the_utils.setting.set_seed(seed: int = 4096) → None[source]

Set random seed.

Note

!!! conv and neighborSampler of dgl are somehow nondeterministic !!!

Set seeds according to:

Parameters:: seed (int, optional) – random seed. Defaults to 4096.

the_utils.setting.set_device(gpu: str = '0') → device[source]

Set torch device.

Parameters:: gpu (str) – args.gpu. Defaults to ‘0’.
Returns:: torch device. device(type=’cuda: x’) or device(type=’cpu’).
Return type:: torch.device

Module contents

the_utils