The DataCacheManager in Kepler
The DataCacheManager (DCM) was designed primarily to "cache" data that
comes from Ecogrid. The design consists of a abstract base class where
the primary function of the derived classes is to implement "how" the
data is to be "download." The data is then dowloaded on its own thread,
whne the download is complete the registered listeners are notified.
Each data item in the cache is stored as a separate file. The
DataCachManager stores the "index" or "directory" of all the items in a
simple XML file. A small separate application the DataCacheViewer will read the index
file and produce a very simple table like representation of the
entries. The contents of any of the Textual entries can be viewed in a
free flowing text control or a table.
The initial usage pattern was for the consumer to ask the DCM for a
specific piece of data or metadata and depending on the type of data or
where the data may be stored the consumer would ask the cache to return
a specific deriied cache object that could handle getting that type of
data. The consumer should never make any assumptions about whether the
data is in the cache or out on the network.
The signature of method call for getting the data in a transparent way
is as follows:
public static DataCacheItem getCacheItem(DataCacheListener aListener,
String aName,
String aResourceName,
String aClassName)
The cache item is identified within the cache manager by the Name, ResourceName, and the ClassName. The Name, is any arbitrary name, the ResourceName, is usually the actual
URI of where the data is to be retrieved from, the ClassName is the type of object
that should be created to retrieve the information. You can always cast
the return object to DataCachItem or to the class of the name you
passed in. The listener will be notified when the data retrival
completes.
For example, here is the code from the Eml200DataSource that is search
metadata in Metacat:
_cacheMetaDataItem
= (EcogridMetaDataCacheItem) DataCacheManager.getCacheItem(this,
"MetaData " + getRecordId(), getEndpoint(),
EcogridMetaDataCacheItem.class.getName());
_cacheMetaDataItem.setEndPoint(getEndpoint());
_cacheMetaDataItem.setRecordId(getRecordId());
_cacheMetaDataItem.start();
Here the call is saying that it wants to have returned to it an object
of class EcogridMetaDataCacheItem
and the the "endpoint" and "recordid" are used to construct the "final"
URI that used to download the data. The "start" method call is for the
thread and this kicks off the asynchronous retrieval on the thread. You
can also call "doWork()" and it will not create a thread and perform
the retrieval sychronously.
NOTE:
When using asynchronous retrieval no assumptions can be made about when
the consumer will be notified via the listener, so this can cause
problems with timing and the retrieval of other information. For
example, if you need two different datasets and "kick off" two
retrievals, you can never be sure of the order of their return and the
order may be different depending if they are in the cache or out on the
net.
Note that there are helper methods on the DCM for retrieving objects by
a named/value pair.
Generic "Look up"
Looking up an item without having it be created is as follows:
public static DataCacheItem
lookupCacheItem(String aName,
String aResourceName,
String aClassName,
boolean aIgnoreResOnSearch)
The last argument indicates that the look up should NOT match on the ResourceName.
The DataCacheItem
The DataCacheItem (DCI) is responsible for the thread that does the
work to get the data, the loading and saving of the data into a file,
and the generation and consumption of the XML for its directory entry.
The DCI has several preset attributes that are used to describe it
within the cache directory. More recently the each DCI can have one or
more "named" attributes with text values. These are automatically saved
and loaded for any derived class. The DCI also provides some abstract
methodsfor derived classes that enable them to have their own "preset"
attributes be read and written as actual attributes of the class.
Here are the methods for getting and setting of the named value pair
attributes:
public setAttr(String
aName, String aValue)
public removeAttr(String aName)
public String getAttr(String aName)
(Currently is does NOT escape the name or the value, I may want to add
that.)
Using
the DCM File Cache Features
There are times within Kepler that an actor may construct or obtain
some data that should be cached either temporarily or permanently.
There are a handful of methods and a support derived class to assist
with this functionality.
The motivation is that there may be a dat in memory that needs to be
saved into the cache as a file, or there may already be some data on
the disk that needs to be "cached."
In general there is a "putFile" and a "getFile" method, but there are
also other methods to assist when working with files. When working with
files and the DCM there is still a notion of the "ResourceName" but it
is really just an additional way to identify a file. If the
ResourceName is not null then it is used in addition to the Name arg
when looking up and identifying a file caache item.
External Files
A file that is already on the disk can be physically copied into the
cache or merely referenced by the cache.
To "copy" the file into the cache simple
call the "putFile" indicating the file should be copied:
item =
DataCacheManager.putFile(physicalFileName,
"logical filename",
"file type",
DataCacheFileItem.COPY_FILE_TO_CACHE);
(After the file is copied the "getFileType()" will return
INTERNAL)
Note that the caller is responsible for deleting the external file
after this call, but before doing so the caller should check the return
value of "getStatus()" to make sure it equals "CACHE_COMPLETE" or it
can check the "isReady()" ready method for true.
If the consumer (or caller) want to just reference an external file
then it can put a file into the cache using "EXTERNAL" for the type
argument.
item = DataCacheManager.putFile(physicalFileName,
"logical filename",
"file type",
DataCacheFileItem.EXTERNAL);
This is not recommended from the
standpoint that a user may delete or
move the file.
Creating Cache Files from Memory Data
If the caller has data that needs to be persisted then it can call:
DataCacheFileItem
createFileItemWithBytes(String
aLogicalName, String aFileType, byte[]
aData)
Finding File Items in the Cache
The original API design of the cache was use the Name, ResourceName, and ClassName to do the lookup, if the
item did not exist then the ResourceName was used to obtain the data.
Calling getCacheItem for a file will work, but if the item is not
already in the cache then DataCacheFileItem will have the status of
"CACHE_EMPTY."
The best and most appropriate way to retrieve a DataCacheFileItem is to
"getFile" which uses the Name and ResourceName to look up the item and
return it or return "null" if it doesn't exist.
public
static DataCacheFileItem getFile(String aLogicalName, String aFileType)
or
public
static DataCacheFileItem getFile(String aLogicalName)
To look up an item by name AND a named/value pair use:
public
static DataCacheFileItem getFile(String aLogicalName, String aAttr,
String
aValue)
This approach enables you assign some metadata to the object such as a
description of its file type.
Or you could have it return a "list" of items that match a named/value
pair:
public
static Vector getCacheItemsByAttr(String aAttr, String aValue)
Note: This will return a Vector of DataCachItems not just
DataCacheFileItems so it is recommended to test for "instanceof" before
cast the object in the vector.
Using a DataCacheFileItem
The DataCacheFileItem has a helper method that returns a java.io.File
object from the physical location of the cache object.
public
File getFile()
Removing
Items from the Cache
To remove an item from the cache:
public
void removeItem(DataCacheItem aItem)