Home > Integration > Setup Amazon S3 to Liferay with data migration

Setup Amazon S3 to Liferay with data migration

We slightly introduced Amazon S3 and its basic integration with a clean Liferay installation in a previous article. What should you do, if you have already uploaded files in Liferay and you want to move them from local storage to Amazon S3? Files that are uploaded to Liferay’s local storage will become inaccessible, if you just perform the steps described in the previous article. That’s because the procedure configures Liferay to use Amazon S3 instead of its local storage without moving files that were originally uploaded to the local storage. Attempts to access those files will result to an Internal Server Error. Is it possible to move these files? Yes, it is. Does data migration affects their settings? No, file settings stay untouched. Here is a short intro on how you can migrate files using the Data Migration Tool.

Prerequisites

Have an account on AWS (Amazon Web Services) and basic knowledge of Liferay installation and configuration.

Migration log

Tomcat users have nothing to do. If you are running Jboss 6.1.1 and want to see Liferay log, you have to append

JAVA_OPTS="$JAVA_OPTS -Dorg.jboss.as.logging.per-deployment=false"

to file JBOSS-HOME/bin/standalone.conf.

Prepare for data migration

Liferay has a functionality to migrate already uploaded documents. We have to tell Liferay, where to migrate files. Liferay needs an access key, secret key and a bucket name. Navigate to our previous article, where we described, how to obtain these.

Open (or create) the file named portal-ext.properties in

Tomcat: TOMCAT-HOME/webapps/ROOT/WEB-INF/classes/
Jboss: JBOSS-HOME/standalone/deployments/ROOT.war/WEB-INF/classes

or wherever you have installed your Liferay and paste in:

dl.store.s3.access.key=
dl.store.s3.secret.key=
dl.store.s3.bucket.name=

Paste valid values, save changes and restart your application server (Tomcat, Jboss, ..). The properties file isn’t loaded without the restart and the migration failes.

Migration

We have everything prepared now, migration can start. Open your web browser at your Liferay installation and navigate to Control Panel. From Server menu choose Server administration and go to Data Migration.

Drop-down menu in section “Migrate documents from one repository to another” change to

com.liferay.portlet.documentlibrary.store.S3Store

and Execute.migrate3

Liferay Portal temporarily changes itself to maintenance mode, when the migration starts. You can see, how many files are in your document library. All of these files are moved to Amazon S3 server.

migration4

Redirection to the main page of your Liferay Portal means operation completed, files are already moved and Amazon S3 is set as file storage. Document previews and thumbnails aren’t moved. Instead new will be generated automatically in place.

Completion of migration

Liferay uses Amazon S3 as storage only until the first restart of Liferay. To make the  usage of Amazon S3 permanent, we have to tell Liferay to start with Amazon S3 by default. Add to portal-ext.properties (where you also saved login credentials and bucket name):

dl.store.impl=com.liferay.portlet.documentlibrary.store.S3Store

Tests

  • Probably the easiest test of successful migration is trying to access to originally uploaded file(s) in documents library. Choose one randomly and download it. If download works fine, Amazon S3 should be configured correctly in Liferay and data migration seems to be successful.
  • Check that correct files and folders are inside the bucket on Amazon S3 console that we have configured in Liferay. It’s good to know Liferay’s file structure, it’s well described at official Liferay guide in section “Using the File System store” (the same structure is used on Amazon S3 servers).
  • Advanced users should check logs of application server on which Liferay is deployed to see potential errors coming.Log file is located at:
    Tomcat: TOMCAT-HOME/logs/catalina.out
    Jboss: JBOSS-HOME/standalone/log/server.log

Caution

If you don’t set correct values for the S3 connection or if you don’t restart the application server before of the start migration, the migration fails. Navigation through file structure will work but all files will be unaccessible until Liferay restarts. If it happens to you, you have to delete again

dl.store.impl=com.liferay.portlet.documentlibrary.store.S3Store

or change it to your previous correct settings. If you don’t change it, Liferay will try to access these files with new storage settings and files remain inaccessible even after restart! See details at bug no. 36105.

Finish

From this moment, your Liferay installation uses Amazon S3 cloud storage and files that were originally uploaded to Liferay’s local storage are already moved to S3 servers too.

Used resources

Official Liferay user guide, section Server administration: Data Migration

Advertisements
Categories: Integration Tags: , ,
  1. Ashika S Shetty
    07/04/2015 at 8:09 am

    Hi,
    I have installed liferay on Amazon S3 and integrated by following all the steps mentioned by you.
    I am able to access from the Remote Desktop(AWS) by giving localhost:8080 but i am not able to access liferay from my laptop by giving the hostname:8080… Public IP is not static.
    Can you tell me why i am not able to access from my laptop. Is there any steps which I missed out.

    Ashika

    • 09/04/2015 at 4:16 pm

      Hi Ashika,
      thanks for the question. I’ll assume that the hostname you use is proper public DNS name of your instance. So the reason, why hostname:8080 is not accessible, may be incorrect port forwarding setup on EC2 when port for Remote Desktop is forwarded but other ports are disabled by default for security reasons. You can try to find port forwarding settings on AWS EC2 and enable it for the 8080 port. You can read the topic here http://serverfault.com/questions/569626/how-to-set-up-port-forwarding-on-amazon-ec2, especially the Nathan C. comment.
      Does it help?

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: