Author Archives: admin

MintOAI

MintOAI runs on panic server :  http://panic:9000/manager/

resides at /usr/local/MintOAI/fromSvn/MintOAI@panic  and in the git repository

Running the dist.sh script will create a distribution  zip file .

The zip file can  be extracted and then run using the start script

for example :

:~$ cd /usr/local/MintOAI/mintoai-1.0-SNAPSHOT/

:~$ nohup sh ./start &

after deleting RUNNING_PID if exists.

 

 

 

 

mint deployment walk through

For example the name of the mint2 instance would be “demo”.

Database creation

In postgres for example in the server panic.

Execute the lines :

$:sudo su – postgres

$:psql -p 5433

postgres=# create user demo password 'demo' login;

$:\q

at  the console type:

$:createdb -U postgres -p 5433 -E UTF-8 --lc-collate=C --lc-ctype=C -T template0 -O demo demo

 

login again as demo.

Its helpful to be in the directory with the createSchema.sql file.

$:psql -p 5433 -h localhost demo demo

demo=> create language plpgsql

demo=>\i createSchema.sql
demo=> alter schema mint2 rename to demo
That should setup the schema!

 

Mint Customizations

Create a new directory mint2/custom/demo/images and paste the logo.png of the mint instance (it has to be named logo.png).

Create a new directory called “demo” inside the mint2/WEB-INF/custom directory.

Copy the java and jsp folders of an existing project and paste them there . Edit the mint/WEB-INF/custom/demo/java/custom.properties according to the project requirements.

Create a new directory called “demo@oreo” inside the mint2/WEB-INF/deploy directory. Copy deploy.xml, hibernate.properties and log4j.properties from an existing project and paste them there.

Edit the deploy.xml hibernate.properties log4j.properties files inside the “mint2/WEB-INF/deploy/ demo@oreo” directory accordingly.

 

Commit and deploy .

In tomcat deployment folder create a new demo_deploy script accordingly. Run the script and try it.

EUscreenXL publication

Hello,

The EUscreenXL project differs in Publication respect from other projects, as it publishes to 2 different external places and lets the user decide which data he wants to prepare for which way.

1) The data for the “core collection” has to be prepared according to the euscreen schema, either uploaded directly in this schema, via mapping, or directly through annotation with the build in annotator. The Publication option for this is available on a data upload once there is a derived dataset in the correct schema with valid items in it. The unpublish option is available once there is a PublicationRecord available for the configured core collection schema dataset.

During publish, every item gets an euscreenId by means of md5( orginialId + providername ). This is added into the euscreenId fields in the xml ( so the transformed xml is modified!). This modified xml is then queued to be send via http post request to a noterik server (configured in the custom.properties).

2) The data for “europeana aggregation” is published and unpublished via our standard method. It should be recorded in a PublicationRecord as well.

Arne

Solr support

Hello,

The new version of the mint2 system now has build in support for solr. It utilizes either an embedded solr server, if a data directory for solr is configured in the mint.properties, or it uses a Solr Server via a configured http URL. If neither is configured, there is no solr support.

Together with this, there is now automatically Solr indexing on upload, transformation (and re-itemizing). The ItemBrowser widget on the client contains a field that allows the search of the viewed Dataset for labels, full text or complete custom solr query, where you can specify fieldnames and logic operators. A search over multiple Datasets, although off course possible from solr point of view, is not yet integrated into the UI, neither is support for facetted search.

Arne

Schema changes in new version

Hello,

The schema has undergone some changes to accomodate some requirements of the euscreenxl project. Until now, a Publication was understood to be information attached to a DataUpload object and could only contain some status, a time and an item count. This was not very rich information. The logic of the Publication was (and is) part of the customization for a project.

For euscreenxl it was necessary to have 2 ways to export information to an outside system, and this was not easily put into the current schema. So it was decided to extract the Publication information from the Dataset (DataUpload) and shift it into its own record, the PublicationRecord. Apart from status, time and count information, this now contains links to the original Dataset (a DataUpload for the time being) and the actual version of the dataset that was published.

Code was adjusted as far as humanly possible to be still compatible, but in order to use the new functionality to the fullest, some parts might have to be rewritten. The updateSchema script will transfer information from the Dataset table into the PublicationRecord table, but since it doesn’t know which Dataset was actually send away, it uses the original Dataset for this field.

Further changes to the schema include the removal of the now defunct NodeIndex and related tables, code and Objects.

Deploy mint2 on oreo

Hello,

I made the following changes on oreo in order to support mint2 deploy there.

  • 1. I created a keypair with ssh-keygen in /root/.ssh/ named id_gitlab_deploy. The private key has NO PASSPHRASE, so please don’t use the publick key anywhere, where security is essential.
  • 2. I configured this to be an identity of the root in the .ssh/config file.
  • 3. In /usr/local/tomcat-projects/ I created a git subdir and cloned the mint2 repository from gitlab with git clone git@git.image.ece.ntua.gr:stabenau/mint2.git
  • 4. In the gitlab instance I added the public key as deploy key to the project mint2.
  • In order to update mint2 clone you need to be root and to perform a git pull inside the mint2 directory. The typical build and deploy stuff from the svn version should still work, we will transfer it, when everybody is on board with this.

    Track the svn via git

    Until we decide to adopt git, I will track mint2 on SVN in a git repository and copy this to

    git@git.assembla.com:MINT_Platform.git

    and

    https://git.image.ece.ntua.gr/gitlab/stabenau/mint2svn.git

    You can clone this, but you can’t push to this repositories. Specially the Assembla one is just for show. If we move to git, we would use the gitlab instance in the lab to host the reference repository. If you play with Egit you can create branches and push them to gitlab, just to see how it works. This is just for experimenting, it cannot move back to SVN.

    Custom mint2 behaviour

    Fotis and I have different ideas on how to realize the customization of functionality in mint2. Here are the 2 approaches.

  • New functionality is coded into the base version of Mint. It is surrounded by checks for a configuration variable that only exists in customized versions that wants this functionality. If two custom versions share the same customized functionality, they simply use the same configuration variable.
  • New functionality is coded into special class CustomBehaviour which inherits from class custom. On places where you want custom functionality, you make static calls to the Custom class, which reroutes them to the CustomBehaviour class. If you want to share functionality, you have to code it into the base code and call from CustomBehaviour into it.
  • Both approaches work and should not differ too much in performance. Having coded the latter, I believe that the config approach is good, but suffers from config file bloat and difficult maintenance. So it should be used sparingly, and where the configuration is actually a string value and not a selector for different functionality. The disadvantage is, that you need to edit code and not config if you want to change behaviour.

    For the time being, I will use my approach for customizing Solrize behaviour, until I get convinced of the config file approach.

    git vs svn

    Hello,

    Im toying with the idea off learning something new and fancy, specifically the version control git. So I played a bit around, did a checkout of mint2 with svn

    arnes Mac:~ $ time svn co https://svn.image.ece.ntua.gr/repos/mop/mint2

    .....
    Checked out revision 7388.

    real 2m32.016s
    user 0m7.690s
    sys 0m5.221s

    That checked out ONE version of mint2 into my home dir.

    Then I imported the svn stuff into a git repo (black magic) and hosted it on oreo in my home dir. If you have git on your machine you can do:
    arnes Mac:~ $ time git clone oreo.image.ntua.gr:/home/arne/repo/mint2.git

    Cloning into mint2...
    remote: Counting objects: 15077, done.
    remote: Compressing objects: 100% (3394/3394), done.
    remote: Total 15077 (delta 9616), reused 15077 (delta 9616)
    Receiving objects: 100% (15077/15077), 129.61 MiB | 11.07 MiB/s, done.
    Resolving deltas: 100% (9616/9616), done.

    real 0m24.645s
    user 0m6.482s
    sys 0m2.760s

    Thats approximatly 6 times faster (but its local network, I dont know how it will be different on a slow line). But you got THE WHOLE HISTORY of the project, not just one version.

    play around with $ git log
    and $ git log –help

    And then everything you can commit locally (for your own versioning means) or push it to the server to share with others.

    Panos set up a git hosting system, if you like the idea, we could make it official and all learn to git around (no gitting). Of course we would first figure out the basics before we use it on production stuff!!