Python SDK Reference
The first thing you need to do is to download our python package from PyPI, you can install it with pip
pip install picsellia
Here is the several packages that will be installed along the picsellia package, they are used to perform mathematical operations, array or image manipulations and HTTP requests
numpy==1.19.4
opencv-python==4.4.0.44
Pillow==7.2.0
requests==2.24.0
scipy==1.5.3
To connect your code to Picsell.ia, you must initialize our client.
from picsellia.client import Clientapi_token = '4d388e237d10b8a19a93517ffbe7ea32ee7f4787'clt = Client(api_token)
You will be greeted by this nice message (Pierre-Nicolas is my name, you should see your username here, unless you are named Pierre-Nicolas too 👀 ) :
Hi Pierre-Nicolas, welcome back.
You need to instantiate a Datalake object in order to interact with your pictures or datasets.
If you need to interact with your datalake, you must initialize the Datalake class.
It is a subclass of the Client.
from picsellia.client import Clientdatalake = Client.Datalake(api_token=None, # <-- YOUR API TOKENhost="https://beta.picsellia.com/sdk/",)
api_token(string) Your personal API token (Find it in your profile page 🔥)host(string, optional) the URL of Picsellia platform, you shouldn't have to change it
None
This methods allows you to create a Dataset and attach annotation, this is the best way to migrate from our legacy platform to the new one.
datalake.push_dataset(name=None,description=None,tags=["dataset1", "aerial pic"],imgdir=None,annotations_path=None,format=None,only_annotations=False)
name(string, required) the name of the dataset to be created.description(string, optional) the description of the dataset to be created.tags(list, required) the list of tags (str) to be added to your pictures,imgdir(string, optional), specify aimgdirif you already have the pictures on your machine.annotations_path(string, optional), provide the path of your annotations in json format, ifNoneyou will create a Dataset without annotationsformat(string, required) set tolegacyif you want to migrate from our legacy platform to the new one, otherwise see here the accepted format.only_annotations(bool, optional) if set toTruepicsellia client won't upload pictures to your lake, it will look for existing pictures with the same name in your lake to create a Dataset.
None
The picture object allow you to interact with your assets only.
If you need to interact with your experiments, you must initialize the Picture class.
It is a subclass of the Datalake.
from picsellia.client import Clientpicture = Client.Datalake(api_token=None, # <-- YOUR API TOKENhost="https://beta.picsellia.com/sdk/",).picture
api_token(string) Your personal API token (Find it in your profile page 🔥)host(string, optional) the URL of Picsellia platform, you shouldn't have to change it
To upload assets to your lake
picture.upload(filepath=None, tags=[], source='sdk')
filtepath(string or list) Either onefilepathpointing to the asset to upload or a list of path.tags(list, optional) the list of tags to attach to the upload assetssource(string, optional) Specify the source of the upload, default is "sdk"
picture.list()
A list containing the pictures objects for your datalake.
{'pictures': [{'picture_id': '25d76bee-a6d3-43d7-8620-6ff18f7a5557','internal_key': '15288614-bedb-4cab-97c1-23684cf9c761.jpg','external_url': 'GE_121.jpg','creation_date': '2021-02-07','height': 310,'width': 322,'tag': []},{'picture_id': 'd1cf0d96-5c05-4fb4-aa4a-5e90f3c748da','internal_key': 'ed6e12e3-0db3-461f-bcd9-54d48509680b.jpg','external_url': 'GE_55.jpg','creation_date': '2021-02-07','height': 2908,'width': 4800,'tag': []},{'picture_id': '716e45a8-09f6-4ec5-9dd1-29c313ae2cdf','internal_key': 'a9597e66-584d-4568-b1cc-31b951154edd.jpg','external_url': 'GE_309.jpg','creation_date': '2021-02-07','height': 208,'width': 254,'tag': []},{'picture_id': '88d2b82d-2a38-4c30-912b-79474a617072','internal_key': 'c83493f7-ce61-4a0b-8166-54698d071792.jpg','external_url': 'Test85.jpg','creation_date': '2021-02-07','height': 663,'width': 710,'tag': []},{'picture_id': 'd9d4684f-a2d1-4431-93e7-dce352aff471','internal_key': 'b7a8bb6b-d3f2-46d5-9599-dfb9b4d2f1cd.jpg','external_url': 'GE_466.jpg','creation_date': '2021-02-07','height': 462,'width': 520,'tag': []},}
Fetch images with corresponding tags
pictures.fetch(quantity=1,tags=["drone", "coco"],)
quantity(float, optionnal) the percentage of assets to fetch ( 1 meaning 100%)tags(list, required) a list of tags used to search in yourDatalake
The list of all the fetched assets
[{'picture_id': '8b536f4c-c95b-4f5f-afbe-a9f31242a235','internal_key': '51ee5ee9-5176-4e98-b173-0687ed6c7b2f.jpg','external_url': '9999966_00000_d_0000055.jpg','creation_date': '2021-02-07','height': 1050,'width': 1400,'tag': ['drone', 'coco', 'vizdrone']},{'picture_id': '426ce7bd-7535-4fe5-80cd-c41e07f84c99','internal_key': '7f4f1b60-d1bb-4458-b3bb-5f3d01a8f7eb.jpg','external_url': '9999955_00000_d_0000312.jpg','creation_date': '2021-02-07','height': 788,'width': 1400,'tag': ['drone', 'coco', 'vizdrone']},{'picture_id': '320e69fc-964a-478e-b689-05351213578e','internal_key': '5aa4036e-8050-4fef-9c3c-af9ba46db511.jpg','external_url': '9999955_00000_d_0000043.jpg','creation_date': '2021-02-07','height': 788,'width': 1400,'tag': ['drone', 'coco', 'vizdrone']},{'picture_id': 'bed1ddab-7cf1-460b-99a8-c4125612caa3','internal_key': 'f73fedbf-d87e-4483-859a-77a3f8e38702.jpg','external_url': '9999982_00000_d_0000167.jpg','creation_date': '2021-02-07','height': 1050,'width': 1400,'tag': ['drone', 'coco', 'vizdrone']},{'picture_id': '0bc695d1-03bb-48ea-bd89-5bef5bf02c23','internal_key': '98ac39c5-a157-4a5e-bafc-a8399b90f230.jpg','external_url': '9999974_00000_d_0000049.jpg','creation_date': '2021-02-07','height': 1078,'width': 1916,'tag': ['drone', 'coco', 'vizdrone']},]
Once, you have fetched pictures, you can call status method to visualize the number of assets fetched
pictures.status()
Number of Assets selected : 1472
Delete the list of pictures
pictures.delete(pictures=None)
pictures(list, optional) The list of pictures to delete from your lake, ifnonewill delete the latest fetched pictures.
None
Add tags to selected pictures
pictures.add_tags(pictures=[],tags=["tag_to_add"])
pictures(list, optional) The list of pictures to selected from your lake, ifnonewill delete the latest fetched pictures.tags(list, required) The list of tags to add to the selected pictures
None
Add tags to selected pictures
pictures.remove_tags(pictures=[],tags=["tag_to_add"])
pictures(list, optional) The list of pictures to selected from your lake, ifnonewill delete the latest fetched pictures.tags(list, required) The list of tags to delete from the selected pictures
None
The dataset object allow you to interact with your dataset ( annotations, labels, question and answers ).
If you need to interact with your experiments, you must initialize the Dataset class.
It is a subclass of the Datalake.
from picsellia.client import Clientdataset = Client.Datalake(api_token=None, # <-- YOUR API TOKENhost="https://beta.picsellia.com/sdk/",).dataset
api_token(string) Your personal API token (Find it in your profile page 🔥)host(string, optional) the URL of Picsellia platform, you shouldn't have to change it
dataset.list()
A list containing the dataset objects for your account.
-------------Dataset Name: GoogleEarthShipDataset Version: firstNb Assets: 793--------------------------Dataset Name: VizDrone2017Dataset Version: firstNb Assets: 6470--------------------------Dataset Name: FaceMaskDetectionDataset Version: firstNb Assets: 6024--------------------------Dataset Name: TrashDatasetDataset Version: firstNb Assets: 1435-------------
Fetch dataset with its name and version
dataset.fetch(name="myDataset",version="latest",)
name(string, optional) the name ot the dataset to fetchversion(string, optional) the version of the dataset to fetch, if None, the client will fetch latest
The fetched dataset infos
{'dataset_id': 'd5269525-1639-40e5-9de2-fbdd43ad46a1','dataset_name': 'GoogleEarthShip','version': 'first','size': 793,'description': '','private': True}
Create a a new dataset and attach pictures to it, to do so you first need to fetch pictures
dataset.create(name="myDataset",description='',private=True,pictures=[])
name(string, optional) the name ot the dataset to createdescription(string, optional) the description of the dataset to createprivate(bool, optional) If True, your dataset will be accessible to anyonepictures(list, required) The list of pictures to attach to yourdataset
the id of the created dataset
Create a a new dataset and attach pictures to it, to do so you first need to fetch pictures
dataset.new_version(name="myDataset",version='newVersion',from_version='latest',pictures=[])
name(string, optional) the name of the datasetversion(string, optional) the version name of the dataset to createfrom_version(string, optional)The origin version for your new version, if None we'll create a new version from the latest versionpictures(list, required) The list of pictures to attach to your version
None
Add the fetch pictures to a dataset
dataset.add_data(name="myDataset",version='myVersion',pictures=[])
If you fetched a dataset before, you won't have to specify the name and version of the dataset
name(string, optional) the name of the datasetversion(string, optional) the version name of the dataset to fetch if None, we'll takelatestpictures(list, required) The list of pictures to attach to add
None
Delete the a dataset
dataset.delete(name="myDataset",version='myVersion',)
name(string, optional) the name of the dataset to deleteversion(string, optional) the version name of the dataset to delete if None, we'll takelatest
None
List all annotations for a dataset id or a fetched_dataset
dataset.list_annotations(dataset_id=None)
If you fetched a dataset, you don't need to specify a dataset_id
dataset_id(uuid, optional) the identifier of your dataset, you can find it in the top left corner of your dataset page on picsellia
The annotation list
{'annotations': [{'worker': {'user': 1,'is_active': True,'creation_date': '2021-02-03T13:05:11.572763Z'},'creation_date': '2021-02-07','type': 'Rectangle','duration': 0.0,'reviewed': False,'is_accepted': None,'is_skipped': False,'nb_instances': 1,'data': [{'type': 'rectangle','label': 'boat','rectangle': {'top': 94.0,'left': 192.0,'width': 78.0,'height': 59.0}}],'image_qa': [],'shape_qa': []},{'worker': {'user': 1,'is_active': True,'creation_date': '2021-02-03T13:05:11.572763Z'},'creation_date': '2021-02-07','type': 'Rectangle','duration': 0.0,'reviewed': False,'is_accepted': None,'is_skipped': False,'nb_instances': 1,'data': [{'type': 'rectangle','label': 'boat','rectangle': {'top': 211.0,'left': 171.0,'width': 95.0,'height': 68.0}}],'image_qa': [],'shape_qa': []},{'worker': {'user': 1,'is_active': True,'creation_date': '2021-02-03T13:05:11.572763Z'},'creation_date': '2021-02-07','type': 'Rectangle','duration': 0.0,'reviewed': False,'is_accepted': None,'is_skipped': False,'nb_instances': 1,'data': [{'type': 'rectangle','label': 'boat','rectangle': {'top': 158.0,'left': 199.0,'width': 34.0,'height': 71.0}}],'image_qa': [],'shape_qa': []},{'worker': {'user': 1,'is_active': True,'creation_date': '2021-02-03T13:05:11.572763Z'},'creation_date': '2021-02-07','type': 'Rectangle','duration': 0.0,'reviewed': False,'is_accepted': None,'is_skipped': False,'nb_instances': 1,'data': [{'type': 'rectangle','label': 'boat','rectangle': {'top': 112, 'left': 188, 'width': 78, 'height': 101}}],'image_qa': [],'shape_qa': []},]
Add tags to selected pictures
pictures.remove_tags(pictures=[],tags=["tag_to_add"])
pictures(list, optional) The list of pictures to selected from your lake, ifnonewill delete the latest fetched pictures.tags(list, required) The list of tags to delete from the selected pictures
None
Networks are trained architectures that you can either deploy for inference (if available), use to start new experiments and share within your Organization's models.
{'organization': {'name': 'picsell'},'model_id': 'b76ececa-274d-48de-b39e-70cf73941aba','serving_id': 'b76ececa-274d-48de-b39e-70cf73941aba','tag': ['efficientdet', 'd2', 'COCO', 'base'],'private': False,'network_name': 'efficientdet-d2','description': 'This is a real game changer','model_object_name': '','checkpoint_object_name': '','origin_checkpoint_objects': {},'type': 'detection','files': {'config': 'b76ececa-274d-48de-b39e-70cf73941aba/pipeline.config','model-latest': 'b76ececa-274d-48de-b39e-70cf73941aba/0/saved_model.zip','checkpoint-data-latest': 'b76ececa-274d-48de-b39e-70cf73941aba/ckpt-0.data-00000-of-00001','checkpoint-index-latest': 'b76ececa-274d-48de-b39e-70cf73941aba/ckpt-0.index'},'thumb_object_name': 'b76ececa-274d-48de-b39e-70cf73941aba/effdet.png','framework': 'tensorflow2'}
model_id(string) Unique identifier of your modelowner(hash, user_object) The creator of the modelnetwork_name(string) The name of your modeldescription(string) A short description of what your model doestype(string) The type of application for your model, if you want to perform pre-annotation on Picsellia it has to be one of the following (but you can set your own type otherwise):'detection'
'segmentation'
'classification'
organization(hash, organization object) The organization under which your model is storedprivate(boolean) Tells if your model is available for everyone in the public HUB or notframework(string) The framework used for training'tensorflow1'
'tensorflow2'
'pytorch'
tag(list) List of tags to identify and sort your modelsfiles(dict) Dictionnary containing the list of files of your modellabels(dict) Dictionnary of the labelmap of your modelbase_parameters(dict) Dictionnary of the base parameters allowing anyone to reproduce the training or iterate with already existing parametersreadme_text(str) A markdown text containing more informations about your model
If you want to interact with your models, you have to initialize the Network class.
It is a subclass of the Client.
from picsellia.client import Clientnetwork = Client.Network(api_token=None,host="https://beta.picsellia.com/sdk/v2/",)
api_token(string) Your personal API tokenhost(string, optional) The URL of Picsellia platformn you shouldn't have to change it.
List of all the models for an organization
By default it will list the models of your own organization but you can specify the name of another organization where you are part of the team.
network.list(organization="null")
organization(string, optional) Leave "null" for your organization or set it to the name of another organization where you have been invited.
A list containing the models of the chosen organization.
[{'organization': {'name': 'picsell'},'model_id': 'b76ececa-274d-48de-b39e-70cf73941aba','serving_id': 'b76ececa-274d-48de-b39e-70cf73941aba','tag': ['efficientdet', 'd2', 'COCO', 'base'],'private': False,'network_name': 'efficientdet-d2','description': 'This is a real game changer','model_object_name': '','checkpoint_object_name': '','origin_checkpoint_objects': {},'type': 'detection','files': {'config': 'b76ececa-274d-48de-b39e-70cf73941aba/pipeline.config','model-latest': 'b76ececa-274d-48de-b39e-70cf73941aba/0/saved_model.zip','checkpoint-data-latest': 'b76ececa-274d-48de-b39e-70cf73941aba/ckpt-0.data-00000-of-00001','checkpoint-index-latest': 'b76ececa-274d-48de-b39e-70cf73941aba/ckpt-0.index'},'thumb_object_name': 'b76ececa-274d-48de-b39e-70cf73941aba/effdet.png','framework': 'tensorflow2'},{'organization': {'name': 'picsell'},'model_id': 'b76ececa-274d-48de-b39e-70cf73941aba','serving_id': 'b76ececa-274d-48de-b39e-70cf73941aba','tag': None,'private': True,'network_name': 'vizdrone-test','description': 'This is a real game changer','model_object_name': '','checkpoint_object_name': '','origin_checkpoint_objects': {},'type': 'detection','files': {'config': 'b76ececa-274d-48de-b39e-70cf73941aba/pipeline.config','model-latest': 'b76ececa-274d-48de-b39e-70cf73941aba/0/saved_model.zip','checkpoint-data-latest': 'b76ececa-274d-48de-b39e-70cf73941aba/ckpt-101.data-00000-of-00001','checkpoint-index-latest': 'b76ececa-274d-48de-b39e-70cf73941aba/ckpt-101.index'},'thumb_object_name': '','framework': ''}]
This method allows you to retrieve a particular model in order to update it or store some files.
network.get(identifier=None, organization="null")
identifier(string) Either the name or the id of the model you want to retrieveorganization(string, optional) Leave "null" for your organization or set it to the name of another organization where you have been invited
The Network object
This method allows you to upload a file and save it under your model so you can use it later such as checkpoint files or config files.
network.store(name="", path=None, zip=False)
name(string,) the name of the file, please check this page to know about the Picsellia namespacepath(string, optional) the path to the file you want to uploadzip(boolean, optional) set to True if you want to zip your file or a folder to an archive before upload
This method allows you to set the image of your choice as the thumbnail displayed on the platform like below.
network.update_thumb(path=None)
path(string) the path to the image file
This methods allows you to set the labelmap of your model.
network.labels(labels=None)
labels (dict) Dictionnary containing your labelmap (index starts at 1 not 0)
Here is the COCO labelmap as example :
labels = {"1": "person","2": "bicycle","3": "car","4": "motorcycle","5": "airplane",..."86": "vase","87": "scissors","88": "teddy bear","89": "hair drier","90": "toothbrush"}
None
{'id': 'b8995c82-f304-40d2-a91f-7e2bcc5df429','date_created': '2020-12-06T20:09:29.703959Z','last_update': '2020-12-06T20:09:29.703689Z','owner': {'username': 'Pierre-Nicolas'},'project': {'project_id': '9c68b4ae-691d-4c3a-9972-8fe49ffb2799','project_name': 'project 21'},'name': 'prout2','description': '','status': 'started','logging': None,'files': [{'id': 13,'date_created': '2020-12-08T23:31:13.389276Z','last_update': '2020-12-08T23:31:13.388914Z','large': True,'name': 'model-latest','object_name': 'b8995c82-f304-40d2-a91f-7e2bcc5df429/saved_model.zip'},{'id': 14,'date_created': '2020-12-08T23:35:42.964512Z','last_update': '2020-12-08T23:35:42.964274Z','large': False,'name': 'config','object_name': 'b8995c82-f304-40d2-a91f-7e2bcc5df429/pipeline.config'}],'data': [{'id': 6,'date_created': '2020-12-07T13:14:15.846681Z','last_update': '2020-12-07T13:14:15.846467Z','type': 'value''name': 'acc','data': 0.8},]}
id(string) Unique identifier for the objectowner(hash, user object) The creator of the objectname(string) The name of the experimentdescription(string) A short description of the experimentstatus(string) Describes the experiment statedate_created(date) Date and time at which the object was createdlast_update(date) Date and time of the last update of the objectlogging(hash, logging object) The telemetry of the experimentfiles(list, file-asset object) The files stored for this experimentdata(list, data-asset object) The data saved for this experiment
If you need to interact with your experiments, you must initialize the Experiment class.
It is a subclass of the Client.
It is also used to initialize some class parameters such as project_token, id (experiment id) or name (experiment name), so you don't have to specify them when you call a method of the Experiment class.
If you initialize the Experiment class with the id or name of an experiment or the project token of the project that your experiments belongs, you don't have to fill them when calling any method, they are now variable of the class instance !
Please note that when calling an Experiment method, you can either specify the id or the name of the experiment, you don't have to enter both.
from picsellia.client import Clientexperiment = Client.Experiment(api_token=None,host="https://beta.picsellia.com/sdk/",project_token=None,id=None,name=None,interactive=True)
api_token(string) Your personal API tokenhost(string, optional) the URL of Picsell.ia platform, you shouldn't have to change itproject_token(string, optional) the token of the project you want to focus onid(string, optional) the id of the experiment you want to work withname(string, optional) the name of the experiment you want to work withinteractive(boolean, optional) if set to False, disable all prompt that requires user action
None
This methods allows you to retrieve all the information and assets of an experiment. For more explanation on how this method works, please have a look at this page.
experiment.checkout(id=None,name=None,project_token=None,tree=False,with_file=False,with_data=False)
id(string, optional) the id of the experiment you want to work withname(string, optional) the name of the experiment you want to work withproject_token(string, optional) the token of the project you want to focus ontree(boolean, optional) set to True if you want to create training-ready folders (see the training reference)with_files(boolean, optional), set toTrueif you want to download all the files of your experimentwith_data(boolean, optional), set toTrueif you want to retrieve all data assets from your experiment
JSON-like object
{'experiment_id': 'b76ececa-274d-48de-b39e-70cf73941aba','date_created': '2020-12-12T16:26:59.343322Z','last_update': '2020-12-12T16:26:59.343090Z','owner': 1,'project': '9c68b4ae-691d-4c3a-9972-8fe49ffb2799','name': 'my_new_experiment','description': 'dog cat classificaiton with base parameters','status': 'started','logging': None,'files': [],'data': []}
List all of the experiments related to one project
This method needs to know your project token, you can either fill it when initializing the Experiment class or you can fill it when calling the method
experiment.list(project_token=None)
project_token (string, optional) the the token of the project you want to focus
A list containing the experiment objects for your project.
[{'experiment_id': 'b76ececa-274d-48de-b39e-70cf73941aba','date_created': '2020-12-12T16:26:59.343322Z','last_update': '2020-12-12T16:26:59.343090Z','owner': {'username': 'Pierre-Nicolas'},'project': '9c68b4ae-691d-4c3a-9972-8fe49ffb2799','name': 'my_new_experiment','description': 'dog cat classificaiton with base parameters','status': 'started','logging': None,'files': [],'data': []},{'experiment_id': 'b251ca51-ba03-4144-a1e7-9eb15db46984','date_created': '2020-12-06T20:10:19.101950Z','last_update': '2020-12-06T20:10:19.101772Z','owner': {'username': 'Pierre-Nicolas'},'project': '9c68b4ae-691d-4c3a-9972-8fe49ffb2799','name': 'desc','description': '','status': 'started','logging': None,'files': [],'data': []}]
Creates a new experiment for a project
The id parameter is used to iterate from a previous experiment, if you need more details, please look here for an explanation.
experiment.create(name=None,description='',previous=None,dataset=None,source=None,)
name(string, 60 char max.) the name of your new experimentdescription(string, 1000 char max., optional) a short description for your experimentprevious(string, optional) set it to a previous experiment name if you want to iteratedataset(string, optional) the name of the dataset you want to use for your experimentsource(string, optional) set it to <username>/<model_name> to clone the assets of a model from our model HUB or your organization
An experiment object with your created experiment
{'experiment_id': 'b76ececa-274d-48de-b39e-70cf73941aba','date_created': '2020-12-12T16:26:59.343322Z','last_update': '2020-12-12T16:26:59.343090Z','owner': 1,'project': '9c68b4ae-691d-4c3a-9972-8fe49ffb2799','name': 'my_new_experiment','description': 'dog cat classificaiton with base parameters','status': 'started','logging': None,'files': [],'data': []}
From here, in order to run all the following methods, you mst have checkout or created an experiment first.
This methods allows you to update the information of an experiment
experiment.update(**kwargs)
**kwargs
name(string) the new name for your experimentdescription(string) the new descriptionstatus(string) the new status
The updated experiment object
{'id': 'b76ececa-274d-48de-b39e-70cf73941aba','date_created': '2020-12-12T16:26:59.343322Z','last_update': '2020-12-12T16:26:59.343090Z','owner': 1,'project': '9c68b4ae-691d-4c3a-9972-8fe49ffb2799','name': 'updated_name','description': 'dog cat classificaiton with base parameters','status': 'started','logging': None,'files': [],'data': []}
This method allows you to delete one experiment
experiment.delete()
True if the experiment has been deleted correctly
Delete all the experiments for a given project
experiment.delete_all(project_token=None)
project_token(uuid, optional) the project token of the project you want to delete all experiments
True if the experiments has been deleted correctly
Download the annotations JSON for the dataset attached to an experiment
experiment.dl_annotations(type='all')
type(string, required)'all' (default) download all annotations of the dataset
'accepted' download only accepted annotations from the dataset
JSON containing all of the annotations in the COCO format, the JSON is also stored in experiment.dict_annotations
{'id': 13,'date_created': '2020-12-08T23:31:13.389276Z','last_update': '2020-12-08T23:31:13.388914Z','large': True,'name': 'model-latest','object_name': 'b8995c82-f304-40d2-a91f-7e2bcc5df429/saved_model.zip'}
id(int) Unique identifier for the objectname(string) The name of the file-assetobject_name(string) The path to the file in our object storagedate_created(date) Date and time at which the object was createdlast_update(date) Date and time of the last update of the object
Create a file-asset for your experiment and upload the file on our object storage. Please check this page if you need more details about what it does in the background and how to use it properly.
experiment.store(name="", path=None, zip=False)
name(string,) the name of the file, it will be displayed on Picsell.iapath(string, optional) the path to the file you want to uploadzip(boolean, optional) set to True if you want to zip your file or a folder to an archive before upload
List all the files belonging to one experiment
experiment.list_files()
A list of all the file asset object of your experiment
[{'id': 22,'date_created': '2021-02-09T12:32:18.022694Z','last_update': '2021-02-09T12:32:18.022465Z','large': False,'name': 'config','object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/pipeline.config'},{'id': 23,'date_created': '2021-02-09T12:32:18.068900Z','last_update': '2021-02-09T12:32:18.068723Z','large': True,'name': 'model-latest','object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/0/saved_model.zip'},{'id': 24,'date_created': '2021-02-09T12:32:18.112582Z','last_update': '2021-02-09T12:32:18.112410Z','large': True,'name': 'checkpoint-data-latest','object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/ckpt-0.data-00000-of-00001'},{'id': 25,'date_created': '2021-02-09T12:32:18.156945Z','last_update': '2021-02-09T12:32:18.156767Z','large': False,'name': 'checkpoint-index-latest','object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/ckpt-0.index'}]
Delete all the files belonging to one experiment
experiment.delete_all_files()
True if all the files has been deleted correctly
Retrieve one particular file of an experiment by its name
experiment.get_file(name=None)
name(string) the name of the file asset you want to retrieve
A file-asset object
{'id': 22,'date_created': '2021-02-09T12:32:18.022694Z','last_update': '2021-02-09T12:32:18.022465Z','large': False,'name': 'config','object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/pipeline.config'}
Delete a file from your experiment
experiment.delete_file(name=None)
name(string) the name of the file asset you want to delete
True if the file has been deleted correctly
Update the information of a given file
experiment.update_file(file_name, **kwargs)
file_name(string) the name of the file asset you want to update**kwargs
name(string) the new name for your file
A file-asset containing the updated object
{'id': 72,'date_created': '2021-02-09T12:32:18.293746Z','last_update': '2021-02-09T12:32:18.293556Z','name': 'parameters','data': {'steps': 200000,'nb_gpu': 1,'batch_size': 8,'learning_rate': 0.005,'annotation_type': 'classification'},'type': 'table'}
id(int) Unique identifier for the objectname(string) The name of the data-assetdata(hash, data object) The data logged in this objectdate_created(date) Date and time at which the object was createdlast_update(date) Date and time of the last update of the object
Log your results to Picsell.ia, check this page for more details on how to use it and what you can do with logs.
experiment.log(name=None, data={}, type="")
name(string) the name of the data-asset you want to createdata(dict) a dictionary containing the data you want to display on the platformtype(string, optional) the type of asset you are logging, see all the types available here
List all the data-assets belonging to one experiment
experiment.list_data()
A list of all the data-assets object of your experiment
[{'id': 72,'date_created': '2021-02-09T12:32:18.293746Z','last_update': '2021-02-09T12:32:18.293556Z','name': 'parameters','data': {'steps': 200000,'nb_gpu': 1,'batch_size': 8,'learning_rate': 0.005,'annotation_type': 'classification'},'type': 'table'}]
Delete all the files belonging to one experiment
experiment.delete_all_data()
True if all the data-assets has been deleted correctly
Retrieve one particular file of an experiment by its name
experiment.get_data(name=None)
name(string) the name of the data-asset you want to retrieve
A data-asset object
You will only return what is stored in the data field of the data asset, not all the information about the asset.
{'steps': 200000,'nb_gpu': 1,'batch_size': 8,'learning_rate': 0.005,'annotation_type': 'classification'}
Delete a file from your experiment
experiment.delete_data(name=None)
name(string) the name of the data-asset you want to delete
True if the data-asset has been deleted correctly
Update the information of a given file
experiment.update_data(data_name, **kwargs)
data_name(string) the name of the data-asset you want to update**kwargs
name(string) the new name for your data-assetdata(dict) the updated data for your asset, please refer to this page for details on the data objects
A data-asset containing the updated object
{'id': 72,'date_created': '2021-02-09T12:32:18.293746Z','last_update': '2021-02-09T12:32:18.293556Z','name': 'parameters','data': {'steps': 5000000.0,'nb_gpu': 8,'batch_size': 64,'learning_rate': 0.0055,'annotation_type': 'detection'},'type': 'table'}
Logging is the telemetry (if existing) of your experiment. It allows you to record in real time what's happening in the interpreter during an experiment and to monitor it from the platform.
logs(dict) The raw stdout of your interpretersteps(dict) The categories of your logs, useful to navigate inside your logstimestamps(dict) The timestamps for each stepexit_code(dict) The exit code and the timestamp of the end of your run