file-collection

Library to manage large amount of files coming from different sources

Introduction

This library allows to manipulate a large amount of files coming from local or remote sources.

This library allows to have the same code in the browser and in nodejs.

This package allows to create a file-collection than can further be saved as a ium file (zip file containing all the data and an index.json file).

The structure of the zip file is at follow:

• index.json
• data/
  • all the files

Installation

npm i file-collection

Basic usage

Append a browser filelist

import { FileCollection } from 'file-collection';

const fileList; // a fileList resulting from a drag / drop in the browser
const fileCollection = new FileCollection();
await fileCollection.appendFileList(fileList);

// get a zip file that can be reload later
const iumFile = fileCollection.toIum();

// list the content of the fileCollection
for (const file of fileCollection) {
  console.log(file.name);
  console.log(await file.text());
}
Copy

Reload a 'ium' file

import { fromIum } from 'file-collection';

const fileCollection = await fromIum(iumFile);

for (const file of fileCollection) {
  console.log(file.name);
  console.log(await file.text());
}
Copy

Using in a react component

A webSource is a URL to data like for example https://image-js.github.io/image-dataset-demo/index.json:

<MyComponent webSource onchange={(fileCollection) => {}}></MyComponent>
Copy

Inside the component we should:

const fileCollection = new FileCollection();
if (webSource) fileCollection.appendWebSource(webSource);
Copy

If we drag / drop some files we should:

fileCollection.appendFileList(fileList);
Copy

If we drag / drop a 'ium' file we should replace the full fileCollection

const fileCollection = FileCollection.fromIum(iumArrayBuffer);
const state = fileCollection.get('state');
Copy

If we want to save as a ium file:

fileCollection.set('state', myCurrentState);
const toSave = fileCollection.toIum();
Copy

When fileCollection changes we can always retrieve the various files using:

const files = [...fileCollection]; // fileCollection is iterable on its files property
Copy

Referencing files in the fileCollection

Internals - sources versus files

In an instance of FileCollection we have 2 properties:

• sources: an array containing the different data sources
• files: an array containing all the files contained in the different sources

Some source may contain many files. This is the case of the .iumfiles as well as for the .zip file.

License

MIT