How To Interact with MogileFS
We’ll start with introducing the commandline utilities as a method to upload, fetch, and inspect files from MogileFS.
Tools of the Trade
The commandline utils are provided by the MogileFS::Utils package. If you have
mogadm you probably have these guys. Note that this wiki page requires
version 2.19 and higher.
All of the utilities have basic documentation in
pod format. If you want to
mogupload, you could run man or perldoc (depending on your
$ man mogupload ... or... $ perldoc mogupload
That gives you syntax, examples, etc.
Upload and Download
Assuming you have a running MogileFS instance from following InstallHowTo, and have a domain created, you may start shoveling files to and from it.
For review, setting up a domain:
$ mogadm --trackers=host domain list ... review any domains you might already have...
$ mogadm --trackers=host domain add testdomain $ mogadm --trackers=host class add testclass ... and we'll add a class to use later...
First, I recommend setting up a mogilefs.conf file somewhere, so you don’t have to specify the tracker and domain every time. Most of the utilities pull from the same config file, so you can type a bit less.
$ echo -e "trackers = host\ndomain = testdomain" >> ~/.mogilefs.conf $ cat ~/.mogilefs.conf trackers = host domain = testdomain
Now, pick a file to upload. Preferrably something smallish.
$ dd if=/dev/zero of=toastfile bs=1M count=1
We’ll use the “mogupload” and “mogfetch” utilities to have some fun.
$ mogupload --key="/helloworld" --file="./toastfile" ... if all goes well, no output...
While we’ve said “/helloworld” here, a key may be anything. It could be “helloworld” or “UUID:13583589233523589235” or “MD5:ABCDEFetc”. Whatever makes the most sense for your application.
You can also use STDIN.
$ mogupload --key="/hellostdin/" --file="-" < ./toastfile
Then get the file back:
$ mogfetch --key="/helloworld" --file="./toastfilefetched" .. you probably have a ./toastfilefetched file now.
Also with STDOUT.
$ mogfetch --key="/helloworld" --file="-" > ./stdouttoast
Now lets go bananas and make a copy of the file with that new class!
$ mogfetch --key="/helloworld" --file="-" | mogupload --key="/hellocopy" \ --class="testclass" --file="-"
… though if you just want to change the class of a file, there’re ways of doing that from client code.
$ mogdelete --key="/goodbyeworld"
There are other utilities for listing fids, finding metadata information, and
troubleshooting files. See
perldoc MogileFS::Utils for an up-to-date list,
perldoc mogprogramname for examples and details of the utility.
Writing an Application
While using MogileFS as a personal filestore is fun, most of you are writing applications. Since MogileFS is a non-POSIX distributed service, serving files from it takes a little understanding.
Click on to AppExample to see one in action.