Python SDK Reference

Get the Python Package, this is not based on any deep learning framework, but it's built to integrate smoothly in your deep learning process.

pip install picsellia

General Class

Client

The Picsell.ia Client contains info necessary to connect to the Picsell.ia Platform. It provides top-level methods to :

  • Pre-process data for training

  • Download annotations & images

  • Send training logs

  • Send infered images

  • Save weights and SavedModel to Picsell.ia server and push it to the model Hub.

Class Methods

Initializes a Picsell.ia Client.

Client(api_token, host="https://app.picsellia.com/sdk/")

Args: api_token (str): Token key, given on the platform. host (str): URL of the Picsell.ia server to connect to.

Raises: NetworkError: If Picsell.ia server not responding or host is incorrect.

Attach it to the desired project

checkout_project(project_token, png_dir=None)

Args: project_token : Token given in your project settings png_dir : path/to/your/images If None, you will be able to get your pics with dl_pictures().

Raises: AuthenticationError: If token does not match the provided token on the platform. NetworkError: If Picsell.ia server not responding or host is incorrect. ResourceNotFoundError: If the png_dir provided doesn't point to images NotImplementedError: If there are files in the png_dir with unsupported types (png, jpeg, jpg)

Create a new network

create_network(self, network_name, orphan=False)

If the model name exists on the server for this project, we will create a new version of your training. Create all the repositories for your training with this architecture :

your_code.py

  • project_name/

    • images/

    • network_name/

      • training_version/

        • logs/

        • checkpoints/

        • records/

        • config/

        • results/

        • exported_model/

Args: network_name (str): Name you want to give to your NeuralNet orphan (bool) : default as False, if True, will create a Network on our Platform but won't create any directory on your machine, use it to push data on the platform if your model has not been created with Picsell.ia.

Raises: AuthenticationError: If token does not match the provided token on the platform. NetworkError: If Picsell.ia server not responding or host is incorrect.

Checkout a network

checkout_network(network_name, training_id=None)

If the model name exists on the server for this project, we will create a new version of your training. Create all the repositories for your training with this architecture :

your_code.py

  • project_name/

    • images/

    • network_name/

      • training_version/

        • logs/

        • checkpoints/

        • records/

        • config/

        • results/

        • exported_model/

Args: network_name (str): Name of the network you want to check out training_id (int): Id of training you want to check out

Raises: AuthenticationError: If token does not match the provided token on the platform. NetworkError: If Picsell.ia server not responding or host is incorrect. ResourceNotFoundError : If the network name you provided does not exists on our server.

Download Annotations

dl_annotations(option="")

Args: option (str): Export option (accepted or all) option = "accepted" will download only the reviewed and accepted annotations. Raises: NetworkError : If Picsell.ia server is not responding or host is incorrect. ResourceNotFoundError: If we can't find any annotations for that project.

This function is called when checking out a network. Your annotations will be available under the attribute client.dict_annotations

Train Test Split

train_test_split(prop=0.8)

Args: prop (float) : Desired proportion for training set Raises: ResourceNotFoundError If no annotations in the client object @ client.dict_annotations

dl_pictures()

Download all annotated images present in client.dict_annotations to your machine and save it to project_name/images Raises: ResourceNotFoundError If no annotations are present in the client object @client.dict_annotations

Generate label map

generate_labelmap()

Generate the label_map.pbtxt file needed for Tensorflow training @ project_id/network_id/training_id/label_map.pbtxt This function is called when checking out a network.

Raises: ResourceNotFoundError If no annotation in the client If no directories have been created

Send label map

send_labelmap(label_path=None)

Send label/id matching to Picsell.ia Backend. This function is called when checking out a network.

Args: label_path (str) path/to/your/labelmap, if None will send self.label_map

Raises: ValueError If no label map created NetworkError If impossible to connect to Picsell.ia Backend

Send Training Logs

send_logs(logs=None, logs_path=None)

Send training logs to Picsell.ia Backend. You should provide only one parameter, logs or logs_path. If both are provided then logs_path will be the logs sent to the platform.

Args: logs (dict) : dict of logs extracted from tf.events (find how to in the how to section) logs_path (str): Path to the logs saved as JSON.

Raises: NetworkError If connection to Picsell.ia Backend failed ResourceNotFoundError If no network_id in @ client.network_id

Send Evaluation Metrics

send_metrics(metrics=None, metrics_path=None)

Send evaluation metrics to Picsell.ia backend. You should provide only one parameter, metrics or metrics_path. If both are provided then metrics_path will be the metrics sent to the platform.

Args: metrics (dict) Dict of the evaluation metrics metrics_path (str) : path/to/metrics json file.

Raises: NetworkError If connection to Picsell.ia Backend failed ResourceNotFoundError If no json file at metrics_path

Send result images

send_results(_id=None, example_path_list=None)

Send inference results to Picsell.ia Backend

Args: _id (int) id of the training, if not provided, will useclient.training_id example_path_list(list) Optional, list of images path to send to the platform

Raises: NetworkError If impossible to connect to Picsell.ia Backend FileNotFoundEror If no inference examples have been found ResourceNotFoundError If inference have not been saved to client.results_dir

Send Training Checkpoints

send_checkpoints(index_path=None, data_path=None, config_path=None)

Send the index file and the .ckpt to retrieve your training stage or allow user to perform transfer learning from your experiments.

Args: index_path (str) path/to/index_file, If None, will look for file @ client.checkpoint_dir/model.ckpt-{}.index data_path (str) path/to/data_file, If None, will look for file @ client.checkpoint_dir/model.ckpt-{}.data config_path(str) path/to/config_file, If None, will look for file @ client.checkpoint_dir/pipeline.config

Raises: FileNotFoundError No such file at client.checkpoint_dir/model.*.* ValueError If you provided a data_path but no index_path

Send exported model

send_model(file_path=None)

Send exported model for inference to the Picsell.ia platform

Args: file_path (str) path/to/saved_model.pb, if None will look for file @ client.network_id/client.training_id/saved_model.pb

Raises: NetworkError If impossible to connect to Picsell.ia Backend FileNotFoundEror If no frozen inference graph have been generated ResourceNotFoundError If no .pb file saved @ file_path

Send all the training data

send_everything(training_logs=None, metrics=None)

Wrapper function to send data from a training. Will assume default parameters for every function except for the training logs and the metrics. This function will try to send the label map, training logs, metrics, checkpoint, model and results. Args: training_logs: Dict of logs extracted from tf.events (find how to in the how to section) metrics: Dict of the evaluation metrics

Upload a Dataset

create_and_upload_dataset(dataset_name, path_to_images)

Create a Dataset and upload the images to Picsell.ia

Args: dataset_name (str) path_to_images (str) path/to/images/directory

Raises: ValueError if no dataset name or path provided FileNotFoundError if no images found @ path_to_images

Upload annotations

upload_annotations(annotations, format='picsellia')

This one is quite complicated ! let's go to our dedicated page

Tensorflow variable generator

tf_vars_generator(label_map, ensemble='train', annotation_type="polygon")

Generator for tf_records creation

Args: label_map (dict) label_map dictionary loaded from the pbtxt file created withgenerate_labelmap() ensemble (str) train or test annotation_type (str) "polygon" , "rectangle" or "classification"

Yields: width (int) image width height (int) image height xmins (array) left box x coordinates xmaxs (array) right box x coordinates ymins (int) top box y coordinates ymaxs (int) bottom box y coordinates filename ('utf-8') utf-8 encoded filename encoded_jpg (bytes) bytes encoded image image_format (bytes) bytes encoded image extension classes_text ('utf-8') utf-8 encoded classes name classes (array) classes in images (if annotation_type=="polygon") masks encoded image masks for segmentation

‚Äč