Installation
Install from PyPI:
1pipinstallopendal Or install from
:
1condainstallconda-forge::opendal Local Usage
Developer must set two required arguments to work with files locally: - scheme: which should be specified as fs - root: where OpenDAL considers the root of the directory for operations will be.
For example in the following operator: opendal.Operator("fs", root="/foo")
OpenDAL considers /foo to be the root of all paths, and that means that we can access paths inside of /foo without specifying anything else. If /foo happens to contain the file baz.txt, we can simply call .list("/")
We can see this in the following example:
1 2 3 4 5 6 7 8 910111213fromopendalimportOperatorfrompathlibimportPathimportosos.makedirs("foo",exist_ok=True)file=Path("foo","bar.txt")file.touch()op=Operator("fs",root=Path("foo").name)forfileinop.list("/"):print(file)When running we get the following output:
12/ bar.txt If we want full access to our file system, we can specify a root of "/", but note that for any operations you will always need to specify the full path to a file or directory.
Reading
There are two ways to read data using OpenDAL. One way is to use the read method on an operator:
1 2 3 4 5 6 7 8 910111213141516fromopendalimportOperatorfrompathlibimportPathimportosos.makedirs("foo",exist_ok=True)file=Path("foo","bar.txt")file.touch()file.write_text("baz")op=Operator("fs",root=Path("foo").name)data=op.read(file.name)print(data)Yields the following:
1b'baz' Note that the output is bytes, but we can get it as a string by simply calling .decode() on the data we read. All reads with OpenDAL return bytes.
Now lets use the open method on an operator:
1 2 3 4 5 6 7 8 910111213141516fromopendalimportOperatorfrompathlibimportPathimportosos.makedirs("foo",exist_ok=True)file=Path("foo","bar.txt")file.touch()file.write_text("baz")op=Operator("fs",root=Path("foo").name)withop.open(file.name,"rb")asf:print(f.read())This again yields
1b'baz' Again, note that all reads with OpenDAL return bytes, so specifying a mode of "r" will raise an exception.
Writing
Now that we know how to read data, let's replace the Pathlib code above with OpenDAL using the write method:
123456789fromopendalimportOperatorfrompathlibimportPathop=Operator("fs",root=Path("foo").name)op.write("baz.txt","my amazing data".encode())print(op.read("baz.txt").decode())This yields the following:
1my amazing data And again, but using the open method:
1 2 3 4 5 6 7 8 910fromopendalimportOperatorfrompathlibimportPathop=Operator("fs",root=Path("foo").name)withop.open("baz.txt","wb")asf:f.write("my amazing data".encode())print(op.read("baz.txt").decode())This again yields:
1my amazing data Again, note that all writing happens in bytes and a mode of "w" will raise an exception.
Async
OpenDAL supports async operation on all operator methods. One can simply replace the Operator with an AsyncOperator and await on method calls. The below example illustrates this behavior:
Standard API:
123456importopendalop=opendal.Operator("fs",root="/tmp")op.write("test.txt",b"Hello World")print(op.read("test.txt"))print(op.stat("test.txt").content_length)Async API equivalent:
12345678importasyncioasyncdefmain():op=opendal.AsyncOperator("fs",root="/tmp")awaitop.write("test.txt",b"Hello World")print(awaitop.read("test.txt"))asyncio.run(main())