The data module provides functions for loading common datasets and iterating over batches of data.
Types
CIFAR10Dataset = ref object of Dataset
- Source Edit
DataLoader = object dataset*: Dataset batchSize*: int shuffle*: bool
- DataLoader provides an iterator to read batches of data from a dataset. C is the number of image channels Source Edit
MNISTDataset = ref object of Dataset
- Source Edit
Procs
proc `$`(d: DataLoader): string {....raises: [ValueError], tags: [], forbids: [].}
- Source Edit
proc cifar10Dataset(train = false): CIFAR10Dataset {....raises: [Exception, ValueError, OSError, IOError, KeyError, HttpRequestError, LibraryError, SslError, TimeoutError, ProtocolError, ZippyError], tags: [ReadEnvEffect, RootEffect, ReadDirEffect, WriteDirEffect, ReadIOEffect, WriteIOEffect, TimeEffect], forbids: [].}
- CIFAR10 dataset of 32x32 color images in 10 classes per http://www.cs.toronto.edu/~kriz/cifar.html Source Edit
proc mnistDataset(train = false): MNISTDataset {....raises: [Exception, ValueError, OSError, IOError, KeyError, HttpRequestError, LibraryError, SslError, TimeoutError, ProtocolError, ZippyError], tags: [ReadEnvEffect, RootEffect, ReadDirEffect, WriteDirEffect, ReadIOEffect, WriteIOEffect, TimeEffect], forbids: [].}
- MNIST dataset of handwritten digits per http://yann.lecun.com/exdb/mnist/ will download the data to and save a cached copy. Returned shape of each image is [28, 28, 1] Source Edit
proc newLoader(rng: var Rand; batchSize = 0; shuffle = false): DataLoader {. ...raises: [], tags: [], forbids: [].}
- Create a new loader for the given dataset. If batch size is 0 then the batch size defaults to the dataset length. Source Edit
proc normalization(d: Dataset): (seq[float32], seq[float32]) {....raises: [], tags: [], forbids: [].}
- Get per channel mean and std deviation for normalization Source Edit
proc shape(d: DataLoader): seq[int] {....raises: [], tags: [], forbids: [].}
- Shape of one batch of images returned from the dataset. Source Edit
proc shutdown(d: DataLoader) {....raises: [Exception, ValueError], tags: [RootEffect, TimeEffect], forbids: [].}
- Shutdown worker thread Source Edit
Methods
method getItem(d: CIFAR10Dataset; i: int; dout: pointer): int32 {....raises: [], tags: [], forbids: [].}
- Get ith entry and copy data to to data Tensor. Returns label. Source Edit
method getItem(d: Dataset; i: int; data: pointer): int32 {.base, ...gcsafe, raises: [ValueError], tags: [], forbids: [].}
- Get ith entry and copy data to to data Tensor. Returns label. Source Edit
method getItem(d: MNISTDataset; i: int; dout: pointer): int32 {....raises: [], tags: [], forbids: [].}
- Get ith entry and copy data to to data Tensor. Returns label. Source Edit
Templates
template start(d: var DataLoader; dset: untyped; channels: static int; transforms: varargs[ImageOp])
- Start should be called to associate a DatatSet with the loader. If the optional image transforms are set then it will start a new thread to launch the image augmentation process in the background. In this case channels should be set to the number of color channels. Source Edit