Nuxeo DAM offers numerous features for Digital Asset Management (content conversion, comparison, and search by color to name a few). Lately we have been looking “outside the box” by focusing on integrating the Nuxeo Platform with other systems.

Many of our users asked us if it would be possible to have access to their assets stored in a Nuxeo repository within Adobe InDesign. So, we spent some time playing with the SDK provided by Adobe and created a simple yet very useful Nuxeo plugin to do that! (Because that’s how we are at Nuxeo: simple and useful ).

Use Case

Let’s consider the case when Nuxeo DAM is used as an asset repository by an organization’s marketing team while InDesign is used by designers using the assets provided by the marketing team. This Nuxeo plugin will facilitate the collaboration between these 2 teams by allowing the InDesign users to access the Nuxeo DAM content repository.

Technical Challenges

There were few technical challenges we had to overcome to create this plugin:

• Authentication
• Searching assets
• Link persistency

Authentication was the cornerstone of this plugin as we needed to access to assets that are stored in the Nuxeo Platform. We also needed to persist the links, meaning that we had to detect any changes that were made on the asset within the Nuxeo Platform and notify the user using InDesign.

Features Offered by the Plugin

First you need to have Adobe InDesign of course, then once you installed the Nuxeo plugin (explained in the following section) you just have to go through the following menu Window>Extensions> and click on “Nuxeo InDesign Connector”.

The HTML and JavaScript based plugin displays a list of assets in a thumbnail format. Based on the Nuxeo REST API we retrieve assets that the authenticated user has access to (also explained later). You can also search for a particular asset based on the metadata and the indexing capability of Elasticsearch. A click on a thumbnail of the asset will import it into the current InDesign Document. That’s it! From that point you just let your creativity take the lead.

You might think: what happens if my asset is modified in the Nuxeo Platform?

The answer is straight and simple. You will get notified so that you make the decision of updating your imported asset within InDesign. (I will explain later how it works.)

Here’s a short demo of some features of the Nuxeo InDesign Connector:

Plugin Installation

Everything you need for installation is here along with a step by step installation guide in the readme.

Structure of the Plugin

In order to be able to import the project in Eclipse you need to install the Adobe Extension Builder from here. Once you import the plugin you should have something like this:

The main HTML file is index.html. The plugin is based on the Common Extensibility Platform (CEP 5) of Adobe (previously called CSXS). It allows you to create HTML5 based extensions with a rich interface. This is the latest generation of Adobe’s creative suite extension and is simpler to use than the previous ones based on C++ or Flash. You can find more details on the structure of the extension here.

Handling Authentication Issues

We have several solutions in the Nuxeo Platform for the authentication issues. The classic and basic one is the OAuth or the token based method. We chose the token based because the use case was quite similar to the one on Nuxeo Drive, where a user registered on a Nuxeo instance gets a token. Then every time the user makes a request that token is inserted in the header, and voilà!

We used the same mechanism for the InDesign plugin. The first time you use the plugin you need to click on the settings icon and register to a Nuxeo instance by providing the following information:

If everything goes well you will have the following information:

Now you have a valid token registered in your InDesign preferences, and every time you open the plugin you will automatically be connected to the Nuxeo instance above with the token using the following code:

//to save the token into InDesign preferences
app.insertLabel("token", token);
//to retrieve the token from the InDesign preferences
app.extractLabel("token”);

Searching Assets from InDesign

By default we display the latest assets imported in the Nuxeo DAM repository (up to 100 results). We also provide two types of search:

• based on the title of the asset
• based on the folder of the asset

Handling Link Persistency Issues

One of the most challenging parts of the development was to find a solution for link persistency. Assets stored in Nuxeo DAM have their own lifecycle and can go through different types of modification within the Nuxeo Platform or by external tools (such as Adobe photoshop).

We needed a way to detect modifications in an asset in Nuxeo DAM and notify an InDesign user that an asset imported in their current Document has been updated. We chose the solution of polling. We verify regularly whether assets used in the current InDesign document are updated in Nuxeo DAM. For that purpose, when the user imports an asset into his current document we save additional information, such as the unique identifier and the digest (which is a hash of the binary). We then use a fixed time polling loop to check for updates by verifying whether the digest in the Nuxeo Platform is the same or not for each assets present in the current InDesign document. Every time a change is detected, we increment the notification badge. If the user clicks on the sync icon the new version of the asset is imported into their current InDesign document. That’s it!

The post Nuxeo InDesign Connector: Access Content Stored in a Nuxeo Repository from Adobe InDesign appeared first on Nuxeo.

Starting with the Nuxeo Platform 6.0, we’ve integrated MongoDB as a NoSQL alternative to using a relational database such as PostgreSQL.

In recent posts, we referenced our participation in MongoDB Days conferences around the world, our perspective on the use of MongoDB and how dynamic facets of the Nuxeo Platform can provide Metadata Agility with MongoDB.

In this article, we’ll dive deeper into some of the specifics of using MongoDB for storage of metadata with the Nuxeo Platform.

The Nuxeo Platform, being a pluggable system, has a few layers of abstraction. The Document Store represents the storage of metadata and hierarchy relationships, and the generic operations of writing, reading and updating this data. Regardless of how the data is stored the Document Store represents these operations.

Getting more specific, the Visible Content Store represents a multi-table and in practice SQL based storage system. It’s an abstraction of SQL based storage, requiring joins across tables. At the lowest level is the adapter to a particular SQL database to issue the actual SQL call and manage the various particularities of different DB systems.

Since MongoDB is a Document based storage system, we have created a Document Based Store (DBS) abstraction as an alternative to VCS and a MongoDB adapter for the appropriate connector implementation. DBS assumes the storage implementation will be a single record per document and includes an appropriate Transient State manager for multi document transactions.

Relational vs Document Based Storage

Document based storage addresses 3 areas of limitations of SQL based storage of Documents: impedance, scalability, and concurrency.

O/R Impedance Mismatch

Without getting too deep into the theory, in practice, we can just say that it’s an issue of too many tables which limits performance. A Nuxeo document is a set of multiple schema fields which need to be mapped to a set of SQL tables. So, in case of large document objects with multiple schema and therefore many joins, queries requiring the full object become unwieldy. This requires caching and lazy loading in the application. With a Document based storage, a document is a JSON object in a single record in the DB. It is designed to allow large objects in a single record, so we can efficiently load many large objects without compromising. One Nuxeo document matches with one MongoDB document.

Scalability

It’s well known that SQL scale out can be complex since it’s not a native distributed architecture. Most Document based storage systems have been designed to be clustered and scaled out. While MongoDB still requires writes to go to a single primary instance, it’s possible to take advantage of Replica Sets for reads. Sharding is also an option and not difficult to setup.

Concurrency

Every database system is some compromise between availability and consistency even within a particular implementation tweaked with configurations. MVCC determines the degree of locking in a multi-user database for a transaction to achieve a particular level of consistency and availability. Most SQL databases give up a small amount of consistency for acceptable performance by default. In case of a document being spread across multiple tables that require locking to achieve consistency, some availability and performance is sacrificed. With a DBS, a Nuxeo Document CRUD is a single operation, so transactions and locking are unnecessary and there is considerable performance improvement.

Still Needing Transactions

There will be cases where we need to run operations on multiple documents transactionally, such as, when running a workflow and updating node data as well as documents. Nuxeo Automation Chains are also expected to be transactional. The Nuxeo Platform implements transaction control at the application level by tracking a transient state for documents, making changes to objects in memory, and only flushing to DB when another operation in the transaction expects those changes to be present. Additionally, we can implement rollbacks using an Undo Log of the application level transaction. We need actual transactions only in some cases, so most of the time we reap the performance benefits of the non-transactional nature of MongoDB.

Configuring MongoDB with Nuxeo

The capability is included as a bundle with the core distribution and it’s just a matter of changing some configuration properties. First, you’d want to have a running MongoDB accessible from the Nuxeo Platform. Then add the ‘mongodb’ name to the list of templates in the nuxeo.conf configuration file. Upon startup, the Nuxeo Platform will connect and create a Nuxeo DB on MongoDB using default ports and hosts if you haven’t changed them and are running local and a Repository configuration descriptor.

Refer to the documentation to learn more. This is all that is required and nothing in your application has to change or be particular to MongoDB. Abstraction allows this to be transparent.

Use Cases for MongoDB Advantage

Due to the levels of abstractions you have choices of pluggable storage systems for the Nuxeo Platform. There are many factors determining the optimal storage system. SQL will often be the best choice, but as noted, DBS and MongoDB provided certain benefits. They are:

Huge Repository with Heavy Loading

This applies when you have a repository with document counts approaching a billion. This can also happen in cases when every change is versioned which would multiply the document records in the Nuxeo Platform. Combined with write intensive access or recursive updates, an SQL DB could be limited. MongoDB can handle the volume without concurrency or scalability issues.

As noted in another post about benchmark figures, import throughput can be a consistent 4000 documents/second or past 30 million documents. An SQL DB would be limited and throughput would drop early due to the way data is written. In addition, due to concurrency issues, a high number of writes per second doesn’t have the same limitation on concurrent reads as seen in an SQL system.

Large Documents

In situations where each Document is a collection of a vast number of fields, large Document objects are created. This is addressing the impedence mismatch and the number of joins when data is distributed among many tables. In such cases, SQL requires lazy loading of objects after a query because a transaction to load a large number of fragmented objects would be unresponsive. Once the cache of objects is filled performance is good. With MongoDB, no lazy loading or 2 level caching is required because it’s simpler to retrieve these large objects. Improvements up to 15x are possible as shown in our published benchmarks.

Additionally MongoDB is easier to scale out and distribute with ReplicaSets and host on multiple sites for architecture flexibility.

Implementation Choices

While MongoDB is great for the noted use cases, as a system architect we know the perfect solution is often a compromise between competing priorities. You can’t maximize availability and consistency at the same time. Every database choice and configuration is an optimization of this. In the noted use cases we give up some consistency of SQL to handle the large bias toward availability and maximize performance of the usage profile of the system.

At Nuxeo, we strive to provide choices for the varied use cases of our customers and to maximize performance in a variety of environments. Support for MongoDB further expands the ability of the Nuxeo Platform to be the highest performing ECM for a wide range of use cases.

The post Take a Deep Dive into Nuxeo & MongoDB appeared first on Nuxeo.

With the release of Nuxeo Platform 7.4, managing user permissions has become easier than ever before! We have worked a lot around permission or security management and we are happy to provide you with many improvements and new features around it. Let’s take a look at these new features with some examples.

What’s New for a User in Terms of UI

For a great user experience, the Permissions tab is now directly visible on the document even when the user only has read permission. As a user, you can see who has access to the document even if you are not allowed to manage those permissions. This tab is also visible on all documents, not only on folderish documents like it used to be, but also on file objects, note objects, etc.

Here’s an example of user julie who only has read permission but can see this new tab on the document Opportunity-developper.pdf:

User with read permission has access to the new permissions tab

The New Features

Let’s focus on the new features that were added.

One of the many helpful features we added is that permissions can be temporary and the user is notified when the permission is set.

Let’s take an example: We want to give julie the write permission on the Opportunity-developper.pdf document between October 22th and October 29th.

To do this, let’s connect as john to add a New Permission. Select:

• the User / Group that will be granted this permission
• the Right (Read, Write, Remove, or Manage Everything)
• the Time Frame (permanent or date based)

Finally, you can choose to Send an email to notify user. The user will receive the email as soon as the permission on the document is active.

Once added, existing permissions can be edited or deleted. You can also block/unblock inherited permissions.

Adding temporary permission to Julie on the “Opportunity-developper.pdf” document

New permission added to julie

The second feature is Shared with Me on the user dashboard.

Here’s an example: john wants to give read permission to the user david for the document Opportunity-developper.pdf even if david has no access to the Human Resources Department. david won’t be able to access the document through the workspace UI but he can by using the Shared with Me available on the dashboard:  

“Shared with Me” available on the dashboard

“Shared with Me”

Another feature is the new Permissions tab in the Admin Center. It has the ability to look for past and current permissions. This feature is very useful for audit and heavily regulated environments such as in financial and pharmaceutical organizations.

For example, you can find out things like “Who had the write permission on that folder or subfolders between this period and that period?” or “On what documents does julie have permissions” ?

Search for permissions

Last but not the least, you have the ability to Purge ACLs of a given user on a specific part of the repository. This will be very useful in cases where you don’t want the user to have any more access, for instance when a user leaves your organization.

Purge of permissions for julie – Step 1

Purge of permissions for julie – Step 2

Here is a demo of all the features I described above:

The Technical Side

The tab has been rewritten using REST API calls only, providing a very reactive UI. This represents a preliminary work regarding the UI and we may replicate what we did on this tab in the entire product in the coming year.

It is possible to query the documents that have specific ACLs. Here’s an example of a query on documents where bob has been granted a permission:

SELECT * FROM Document WHERE ecm:acl/*1/principal = 'bob'
AND ecm:acl/*1/grant = 1
AND ecm:acl/*1/permission IN ('Browse', 'Read', 'ReadProperties', 'ReadWrite', 'ReadRemove', 'Everything')

Because it is restful, we provide operations and APIs that allow reimplementing what is available.

Also, this has been made using web components. An element called “document-permissions” can be used on any HTML page outside of the Nuxeo Platform (custom HTML page, any portal, any AngularJS website, etc.) to display and use the Permissions screen of a given Nuxeo document. All you have to know is the document ID.

<nx-connection id="nx_connection" url="https://nightly.nuxeo.com/nuxeo/"></nx-connection>
<document-permissions doc-id="8d46d1cf-5ad7-4c27-bba8-ff4021a7c4d1"></document-permissions>

The post Easily Manage Permissions in Nuxeo With Extended ACLs appeared first on Nuxeo.

In a previous blog post, I discussed various storage options for the Nuxeo Platform and specifically support for MongoDB and appropriate use cases. Today, let’s talk about GridFS for binary storage and learn more about the flexibility of the Nuxeo Platform to support various storage and query subsystems to achieve the ideal configuration for your specific use cases.

Let’s take a look at the Nuxeo storage architecture again, in this case configured with MongoDB and GridFS.

You can see that the Nuxeo Platform uses different subsystems for the Document (metadata) and Blob (binaries). A Document is a set of metadata values, various attributes (Facets) as well as referenced binaries, combined in the output from the Repository to the application. A binary in Nuxeo is wrapped by a Document. This allows us to use the most appropriate storage for metadata as well as binaries. Nuxeo Document Store and Blob stores are pluggable, allowing the choice of using these abstractions.

Benefits of Using GridFS

MongoDB users will certainly be familiar with GridFS. By providing a Blob Store implemented by GridFS for binary storage to replace the File System, we are sensibly integrating Document and Blob storage into the MongoDB container.

There are two areas of benefits here:

First, the GridFS capabilities themselves are great. Instead of directly dealing with the File System, GridFS uses the functionality of MongoDB to provide replication, distribution and possible redundancy capabilities. This is possible by breaking every binary into chucks of configurable size, and storing them as individual records in a collection designated for them, while another collection stores records for each file’s information. The GridFS API is built on top of the MongoDB system and handles all the chunking, rebuilding of files and read/write access. Since the files are broken into chunks, you can access the chunks you need for read or write without streaming the whole file. You can also download and stream multiple pieces simultaneously to potentially increase throughput.

The second benefit of combining MongoDB with GridFS is the common management and administration tools and methods. It’s no longer needed to have different backup and infrastructure strategies for database and File System, so it’s effiecient from an administration perspective.

Configuring the Nuxeo Platform to use GridFS

Configuring the Nuxeo Platform to use GridFS instead of the File System is an easy and transparent process. Since the Nuxeo Platform is extension based, it’s just a matter of deploying an XML based configuration to specify the Blob Manager implementation and a couple of properties.

 <extension target="org.nuxeo.ecm.core.blob.BlobManager" point="configuration">
     <blobprovider name="default">
         <class>org.nuxeo.mongodb.blob.GridFSBinaryManager</class>
         <property name="server">localhost</property>
         <property name="dbname">nuxeo</property>
         <property name="bucket">nxblobs</property>
     </blobprovider>
 </extension>

Now when accessing a binary, the system will simply call the same read/write methods on the new implementation. Migration would also be an easy process. The Nuxeo Platform stores binaries in a simple bucket like structure where the only reference is a unique MD5 Checksum based filename. So, if you have binaries already existing in the Nuxeo Platform on the File System, you can simply copy and dump into GridFS and switch the Blob Manager. This is transparent to the application because only the ID is passed to the Blob Manager.

The GridFS package is available on the Marketplace package – MongoDB GridFS Storage

The post Binary Storage with MongoDB and GridFS appeared first on Nuxeo.

We⁽¹⁾ recently had this question: “How easy would it be to implement a feature where a user can build an ordered set of icons and synchronize it with their Desktop?”. The use case was about building toolbars for custom software. As you might have guessed, the answer to this “How easy” question is, of course, “Extremely easy”. Instead of just answering “It’s easy”, we actually built it with Nuxeo Studio (which is why it’s so easy). Nuxeo Studio is the coolest configuration tool in the universe!

I have created a short video showing what we built. I’ll explain the steps in detail after you have watched the video. As usual, since this blog is in the “Nuxeo Studio Rocks” series, we have a playlist starting with the famous Queen’s We Will Rock You⁽²⁾. (Listening to the music while reading the blog (or watching the videos) should be mandatory, but it is not )

Here’s the video:

As you can see, we have four main parts here:

1. Find images
2. Order the list (using drag and drop)
3. Approve/Reject
4. Synchronize with Desktop

Let me walk you through this list.

Step 1: Find Images

For this step, even though the search screen was built using Nuxeo Studio, we are mainly using native, out-of-the-box Nuxeo features:

• Full text search
• Select 1-n images
• Click the “Add to Collection” button and:
  • Create a new collection
  • Add to this collection

Step 2: Order the List

This step requires the wonderful nuxeo-palette plug-in written by The Great Fred. (We have already used nuxeo-palette in a previous blog, so you might be familiar with it).

First, we must install this plug-in on our server. Then, we want to provide the UI in order to allow the user to drag-drop and organize the images. What we have here is actually a perfect example of how cool Studio is: It knows nothing about this “palette.xml” widget, but still lets us use it!

Now, let’s create this “Palette” tab that you saw in the video:

0. Playlist song#2: Status Quo’s Rockin’ All Over the World.
1. Add a new “Tab” and name it “Palette”.
2. We want this tab to be displayed only for a Collection and nothing else. So, click the “Activation” subtab, and set “Current document has facet” to “Collection”.
3. Now, go back to the “Definition” subtab:
   • Set the “Label” to “Palette”⁽³⁾, and the order to -1 because we want this tab to be the first one displayed when the collection is open.
   • Remove the default row and add a new one. Select the one that has a single column.
   • Drop a “Template” widget. It is located in the “More Widgets/Advanced Widgets” part of the list:

• Now, here is the cool feature: In the Layout Widget Editor, scroll down, expand “Custom Properties Configuration”, set the value to “template” and the key to “palette.xhtml” (keep in mind that they are case sensitive), save and voilà (no need to import “palette.xml” in the “Resources” part of your project):

Once you deploy the project, you will have the “Palette” tab allowing the user to organize the documents of the collection as they want, using drag and drop.

Step 3: Approve/Reject

Our user is happy with the toolbar and now it’s time to send it for validation.

All this was done – of course – in Nuxeo Studio. Let’s not go into details here and just link to the features:

1. First, we create a workflow named “PaletteValidation”. It is a classic approval workflow where a user can reject/approve the asset. If the user rejects it then the flow returns to the first user.  The first user must then change the icons and/or change the order (based on the manager’s comments, for instance). In our case, assignment to “manager” (and possibly back to the workflow initiator) is done in each task.
2. Then, we create a User Action. It’s a toolbar button displayed only for documents with the “Collection” facet. (We could be more user friendly here. For example, by making sure a workflow is not already running for this collection. But when we are building a demo or just showing an example, we can avoid too much details.)
3. This User Action runs a basic automation chain which starts the workflow:

Now, when the manager approves the palette, we want the palette to be synchronized with their Desktop and it must keep the order of each image. To keep the order, we decided to add a prefix or a number to each file. This means we can’t simply ask Nuxeo Drive to synchronize the collection itself (which it could do) because Nuxeo Drive uses the file names. We don’t want to change the file names in the Nuxeo repository because the same image may be used in different palettes (renaming one will change the order in other palettes too). So, instead we decide to copy the Documents and rename the copies. We created the “Approved Palettes” folder in which we will copy and handle each image.

Since it requires loops and tests, it is a perfect use case for Scripting Automation. As I am a big JavaScript fan, I have a small tendency to see every use case as requesting Scripting Automation. But the point here is that, in this case, it is true (which does not mean it is false in other cases!). The fact that nuxeo-palette’s Services.GetPaletteItems operation returns a JSON string also helped make this decision. The algorithm is as follows:

• Use nuxeo-palette’s Services.GetPaletteItems operation to get the ordered list of items.
• Create a new Folder in the “Approved Palettes” folder (all this is hard coded for a demo. In a production environment, you will write something more flexible).
• In this folder, copy every Document in the ordered list.
• For each copy, change the title and change the file name, making them start with a number.
• (the whole script is in a few lines)

Step 4: Synchronize with Desktop

This step is actually part of the same Automation Scripting Chain.

After all the renamed copies have been created, we want to make the Desktop Synchronization automatic (so that the user does not need to manually activate it). For this, we can directly call the NuxeoDrive.SetSynchronization operation and we are done with this workflow.

Here is the final script:

// A *lot* of assumptions and hard coded values. This is for a demo.
function run(input, params) {
  var APPROVED_PALETTES_FOLDER_PATH = "/default-domain/Approved Palettes";
  var docListStr, docListJson, approvedPalettesFolder, approvedPalette, doc, docCopy, blob, i;
    
  docListStr = Services.GetPaletteItems(input, {});
  if(docListStr !== "") {
    docListJson = JSON.parse(docListStr);
    approvedPalettesFolder = Repository.GetDocument(input, {'value': APPROVED_PALETTES_FOLDER_PATH});
    
    approvedPalette = Document.Create(approvedPalettesFolder, {
      'type': "Folder",
      'name': input.title,
       'properties': "dc:title=" + input.title
    });

    i = 0;
    docListJson.forEach(function(oneObj) {
      i += 1;
      doc = Repository.GetDocument(input, {'value': oneObj.id});
      blob = doc.getPropertyValue("file:content");
                 
      docCopy = Document.Copy(doc, {
	'target': approvedPalette,
	'name': formatInteger2Zeros(i) + "-" + doc.title
      });
      
      docCopy = Document.SetProperty(docCopy, {
	'xpath': "dc:title",
	'save': false,
	'value': formatInteger2Zeros(i) + "-" + doc.title
      });
      
      docCopy = Document.SetBlobName(docCopy, {
	'name': formatInteger2Zeros(i) + "-" + blob.getFilename(),
	'save': true,
	'xpath': "file:content"
      });
    });
    NuxeoDrive.SetSynchronization(approvedPalette, {'enable': true});
  }
  return approvedPalette;
}

function formatInteger2Zeros(inValue) {
  if(("" + inValue).length > 2) {
    return inValue;
  }
  return ('00' + inValue).substr(-2);
}

JavaScript in Automation makes my day every day. And thanks to Amazing Vlad for making the JavaScript Editor in Studio more and more performant and easy to use!

(1) The Nuxeo Solutions Architects Team. Just “The” for people who know us.
(2) Not sure it’s easy to watch this serious video while listening to this song. This song, somehow, forces you to stand up and play the rhythm.
(3) Or Whatever you Want. It will also work with “Toolbar”, “Icons”, “Free Beer”, etc.

The post Nuxeo Studio Rocks! Find, Order and Synchronize a Set of Images appeared first on Nuxeo.

As we work on various use cases for the Nuxeo Platform, we always come across requirements that give us an opportunity to introduce new features or enhance the existing ones. Today, I will talk about one such requirement: Limiting the download of attached files in a Document.

Let’s extend the requirement by adding the condition that a user should be able to download only some of the attached files of a Document.

The perfect example is a picture or image file. When you upload a picture in the Nuxeo Platform, additional formats will be automatically generated (most of the time with lower resolutions). It is fully customizable and there are a few default formats. Here is an example of what is generated from a Photoshop file.

It is very convenient that the Nuxeo Platform generates these additional formats but maybe you want to restrict some users from downloading the original format or maybe even the original-size JPEG. This is where we will use the new feature, File Download Security, available in our latest release Nuxeo Platform LTS 2015.

The File Download Security Documentation explains this feature very clearly with examples. There is a new extension point that allows us to define a scriptable permission to decide if the download is authorized or not.

Here is the condition that I have set up (directly in the XML extensions of my Studio project). The condition is that users from the group “LimitedDownload” cannot download the main file or the original-size JPEG:

<extension target="org.nuxeo.ecm.core.io.download.DownloadService" point="permissions">
   <permission name="limited">
      <script>
         function run() {
            if (CurrentUser.getGroups().contains("LimitedDownload")){
               if (XPath == "file:content" || XPath == "blobholder:0" || XPath == "OriginalJpeg:content" ) {
                  return false;
               }
            }
            return true;
         }
      </script>
   </permission>
</extension>

That’s it! Users from the “LimitedDownload” group will now get a permission denied error when accessing the restricted file. Now, we are working on improving the UI for the response.

Another thing I like about this is that this is a low level security (not UI based), so it will work the same through API calls. In my example, I did not restrict all the ways to download. For instance, our restricted user can still export the files as a zip folder and get all the formats at once. Let’s make it your homework assignment You can try it out and let us know what you think.

The post Introducing Download Permissions in Nuxeo appeared first on Nuxeo.

We recently looked at how the extensibility of the Nuxeo platform facilitated the use of MongoDB and GridFS for the back end storage of documents and binaries. Due to the modular nature of the platform, it’s just a matter of specifying a different implementation with an extension configuration.

Today, let’s focus on binary storage options, but more specifically, a new feature to expand on storage options further. The FilesystemBlobManager, now included in the Nuxeo Platform, allows referencing binaries outside of the repository from Nuxeo documents without the need to copy into Nuxeo storage and yet making them completely transparent to the application and user.

Architecture

Firstly, an understanding of the Nuxeo binary storage architecture would help put this in context. A Nuxeo document is a composite of metadata as well as zero or more associated binaries (as illustrated in the architecture diagram below). Focusing on the Blob (binary) storage side of the repository, the Blob Store encompasses all operations of storing a binary and returning it further downstream based on an ID. Then, as we get more specific and concrete, a Binary Manager implements a Pluggable Persistence Engine. This can be a choice of various implementations, such as, using AWS S3, an SQL DB or having built in AES Encryption on the stored file.

Note the FileSystem BlobProvider in parallel with the default.

Focusing more specifically on the most commonly used default Binary Manager, which uses the file system, there are also choices in how and where on the file system the Blob is stored and retrieved. The BlobProvider implementation handles that. Normally by default, during imports or uploads, binaries are renamed to a unique ID and copied into a Nuxeo managed file system location with a simple flat structure. The binary is reference by ID alone which eliminates the need for keeping a hierarchy with directories for name conflicts. The binary is referenced from a document just by its unique hash based ID.

Use Cases

There are some cases when a copy is not desirable, so we provide another BlobProvider implementation to create documents in a Nuxeo repository without making a copy but by keeping a reference to the original associated binary. Instead of a copy and renaming into the binary store a full link is stored which references the location. Documents are created normally and all this is transparent to the user. All the previews and thumbnails access the original file and are stored in the default binary store. Users see no difference in behavior except the prevention of writing or modifying through the Nuxeo Platform as the use case dictates.

One such use case is large volumes or sizes of binaries that would take a long time to import but should still be referenced and indexed in the Nuxeo repository. For example, this can be the case with existing video content that uses an existing process of upload into a particular location. This could then be potentially edited and accessed directly for consumption outside of the Nuxeo repository with the storage medium optimized for it. Video content sometimes has such an external optimized workflow.

Another use case would be if there are existing external systems that need to read binaries directly from an existing binary location. You can index and use binaries in the Nuxeo Platform with all its capabilities, without having to potentially change the way existing systems rely on a location or organizational scheme to access binaries.

Usage

Now, let’s look at how to use FilesystemBlobProvider out of the box as well as its future expectations. There are a few different ways of importing files and creating documents in the Nuxeo Platform. The UI allows drag and drop, importing sets of files or explicitly choosing a document types, which creates documents wrapping a binary. Some of these methods are meant to be automatic and therefore must rely on the one default BlobProvider type. We need a way to explicitly create certain documents using FilesystemBlobProvider instead of default. The two methods are Bulk Import configuration and an Operation definition.

The bulk import using the Bulk Document Importer package is a server operation. It’s kicked off by a REST call or by using a UI for configuration, and having a specified Document type, batch size, source and destination. As it’s a server side process, you’d reference a location which the server can access by path. This is the best method for migrations or larger import processes.

When deciding to import/create using an FilesystemBlobProvider and reference binaries directly, we wrote a custom importer for this example to illustrate its usage. A bulk import will now use a different DocumentModelFactory implementation which creates documents with this FilesystemBlobProvider instead of default and the binaries in those documents are file system references. As you can see, it’s only for those you chose to import and your repository can include a combination of different methods of binary storage. This doesn’t have to affect the whole server.

As an addition or the alternative, it’s just as easy to implement a custom Operation which will take the source path of the binaries and create the directory structure for wrap. This can now be called by a user method with input, by a REST call, and other external applications. Those who are familiar with  Nuxeo Studio or development by API will be able to take advantage of this nicely.

In either case, previews and thumbnails will be stored in the default binary store and not be added to your source location. This does not affect the standard UI import and is generally the preferred configuration because a UI import is referencing the client’s machine and the client won’t have the same path as the server does to reference binaries. In the described intended use cases you won’t likely be needing to modify the binary upon import because that would require a copy to the Nuxeo Platform for temp storage, thereby defeating the main purpose of this feature. You can build listeners to keep the thumbnails and previews synchronized to the externally referenced file if you expect modifications.

Implementation Example

The following configuration was done to implement and deploy an importer using FilesystemBlobProvider.

First, you have to register the FilesystemBlobImporter with an ID and the source root location. You can deploy multiple configurations if you have multiple mapped drives for example. The “fs” is the default ID that the example importer will use.

<extension target="org.nuxeo.ecm.core.blob.BlobManager" point="configuration">
    <blobprovider name="fs">
      <class>org.nuxeo.ecm.core.blob.FilesystemBlobProvider</class>
         <property name="root">/Users</property>
         <property name="preventUserUpdate">true</property>
    </blobprovider>
</extension>

Now you extend DocumentModelFactory and implement a new Factory, FileSystemDocumentModelFactory in this case, which creates the document. Here we will show the override method. The key area is retrieving the BlobProvider by ID as defined above.

@Override
public DocumentModel createLeafNode(CoreSession session, DocumentModel parent, SourceNode node) {

    File file = node.getBlobHolder().getBlob().getFile();
    BlobInfo blobInfo = new BlobInfo();
    blobInfo.key = file.getAbsolutePath();
    Blob blob = ((FilesystemBlobProvider)Framework.getService(BlobManager.class)
        .getBlobProvider("fs")).createBlob(blobInfo);
    DocumentModel doc = Framework.getLocalService(FileManager.class)
        .createDocumentFromBlob(session, blob, parent.getPathAsString(), true, file.getName());
    return doc;
}

Then you deploy and include an xml extension which will use this importer instead of the default one.

<extension target="org.nuxeo.ecm.platform.importer.service.DefaultImporterComponent" point="importerConfiguration">
   <importerConfig sourceNodeClass ="org.nuxeo.ecm.platform.importer.source.FileSourceNode" >
      <documentModelFactory documentModelFactoryClass="org.nuxeo.ecm.platform.importer.externalblob.factories.FileSystemDocumentModelFactory" />
   </importerConfig>
</extension>

Now your bulk imports use the FilesystemBlobProvider and don’t copy the source binary. You can reference the documentation for running the Bulk Importer here. Additional resources about Binary Store configuration can be found here and here.

We plan on providing more of these pre-defined implementations as runtime choices and continue to increase configurability as well. Any feedback on the most common use cases and feature requests is welcome!

The post Referencing Existing File System Binaries in a Nuxeo Repository appeared first on Nuxeo.

Most of you might already be familiar with the Nuxeo Diff plug-in. It displays an interface comparing two documents or two versions of a document and shows the differences between them. Thibaud (our Solutions Architect and Chief Blogs Officer ) wrote in details about this plugin in an earlier blog.

Today I will talk about a new plug-in available in Nuxeo Marketplace, Nuxeo Diff Pictures. Using this plug-in, you can go further by comparing two pictures:

• Of the same format and same dimensions
• Or of different dimensions and/or picture format, leveraging ImageMagick capabilities

I will walk you through the installation process and show some cool examples of what Nuxeo Diff Pictures can do.

Installation

This plug-in can be installed like any other package from the Nuxeo Marketplace or from the Admin Center. Also, if you install the “Nuxeo Diff Pictures” plug-in from Marketplace, the “Nuxeo Diff” plug-in will be installed too if it was not already there.

Nuxeo Diff Pictures plug-in requires the Nuxeo DAM plug-in (to be able to manage pictures of course!).

How You Can Use It

Step 1: We need two pictures that we can compare. So, download a picture, version it and then modify your picture. Let’s take two examples:

• Add a new object and create a new version (figure 2 below)
• Convert your image from JPEG to PNG (figure 3 below)

Step 2: On the picture document’s History tab, click on the Archived versions sub-tab.

Step 3: Select the two versions you want to compare by checking the corresponding boxes and then click on the Compare button.

The fields for which there are differences between the two versions are displayed in a table.

Figure 1: Comparison of Properties

Step 4: To visualize the changes in the picture, click on the icon on the right.

A window pops up showing what’s been changed between the two versions.

The dialog also lets the user set-up a “Fuzzy” comparison parameter (to remove noise when comparing JPEGs for instance). The user can also choose the colors used to highlight the differences.

Figure 2: Compare pictures, same format

These options are not available when comparing pictures of different formats/dimensions, but the result in this case is amazing too:

Figure 3: compare pictures, different formats: JPEG and PNG

Give it a try and tell us what you think!

The post Display Differences in Pictures Using Nuxeo Diff Pictures appeared first on Nuxeo.

Keeping up with our trend to make the Nuxeo Platform better and more powerful, we have introduced some cool features in our latest release, Nuxeo LTS 2015. One of these cool features is that you can now configure the Nuxeo Platform to forward all downloads directly to a cloud blob storage endpoint, such as Amazon S3 or Microsoft Azure. This means that your server will be free during the download and it will be able to handle a lot more download requests, thus scaling you blog download rate significantly!

Today, I’ll show you how you can do that with Microsoft Azure. In my next blog, I’ll talk about how to do the same using your existing S3 install.

Configure the BinaryStore Manager with Azure

You can get a global view of the module configuration and help from the documentation about Microsoft Azure binary manager or documentation about Amazon S3 binary manager.

To start the configuration, let’s use the passwords encryption in your configuration file. It allows you to encrypt any configuration entry in your nuxeo.conf file, and in our case, can hide your access tokens from the bad guys!

Assuming you are currently using a registered Nuxeo instance, here’s what you can do:

 # Install Microsoft Azure marketplace package
 $ nuxeoctl mp-install microsoft-azure-online-storage
 
 # Add the encrypted value of nuxeo.storage.azure.account.key
 $ nuxeoctl config --set --encrypt nuxeo.storage.azure.account.key V3rYUns3Cur3Ent4Y/k3YCryPt3d/NiSamVMdgEqPN+jINb3Bg81Two06LJEtcpM/Oyl1KcuDC7E3/Z==
 # And why not doing the same with the account name?
 $ nuxeoctl config --set --encrypt nuxeo.storage.azure.account.name Als0Crypt3d0559843256027
 
 # The container name can be in clear text (Do not be too paranoid).
 $ nuxeoctl config --set nuxeo.storage.azure.container my-container

Your blobs are now stored in a scalable and very efficient cloud storage engine and your access tokens are secure.

With this configuration, when a user tries to download a blob for the first time:

• the file is downloaded from the cloud storage
• stored in the binary manager local cache for further use
• then returned to the client as quickly as possible

But, we want to achieve a very large number of downloads and we can do much better than this! Let’s see how.

Increase Your Download Rate

Let’s significantly increase the amount of downloads that we can handle by enabling the new direct download feature. With this configuration, when a blob download is requested:

• Nuxeo responds to the client with a redirect to the cloud blob store
• The client downloads the file directly from one of the cloud storage facilities

You might be wondering: What about the security!? I do not want my blobs to be accessed by everyone. Don’t worry, it won’t happen! The request is signed to allow the download on the given URL only for a small amount of time, then you’ll need to get another url.

All you have to do is add these lines to your nuxeo.conf file and you are all set.

 # If you are using Azure storage
 nuxeo.storage.azure.directdownload=true
 
 # Or S3
 nuxeo.s3storage.downloadfroms3=true
 
 # You can also change the expiration time of the direct link with (in seconds)
 nuxeo.storage.azure.directdownload.expire=1800
 nuxeo.s3storage.downloadfroms3.expire=1800

Better and Faster with a Content Delivery Network (CDN)

Here’s something else you might be thinking now: Ok, that’s sounds great. But all my clients are spread across the world and they may get some latency issues depending on the region.

Don’t worry! These main storage systems (S3 and Azure) have already thought about it, and they allow you to easily bind a Content Delivery Network (CDN) to your cloud storage. I will not discuss how CDN works, but to give you an idea, let’s consider this scenario: Assume that previously with the cloud storage your blob was available from one region only (except if you enable S3 Cross-Region replication, or Azure RA-GRS replication). Using a CDN, your content is automatically replicated in several regions and makes your worldwide users very happy.

Here’s how you can configure the Azure CDN.

Configuring Azure CDN

The configuration is really simple. Just follow the documentation on how to integrate a Storage Account with CDN. Then, add these lines in your nuxeo.conf file:

 nuxeo.core.binarymanager=org.nuxeo.ecm.blob.azure.AzureCDNBinaryManager
 
 # Ensure direct download is enable
 nuxeo.storage.azure.directdownload=true
 
 # Then set the cdn host which is binded to your storage
 nuxeo.storage.azure.cdn.host=nx201500.vo.msecnd.net

Your download requests will now be redirected to the CDN.

Configuring Your Own Content Delivery Network Strategy

If you prefer to use some other CDN or simply your own Proxy to cache content that needs an url rewrite, that’s fine too. It’s definitely possible. Open your prefered IDE and create a new class that extends:

org.nuxeo.ecm.blob.azure.AzureBinaryManager,org.nuxeo.ecm.core.storage.sql.S3BinaryManager or just your own org.nuxeo.ecm.core.blob.binary.BinaryManager.

All you have to do is override the org.nuxeo.ecm.blob.AbstractCloudBinaryManager#getRemoteUri method like we have done in org.nuxeo.ecm.blob.azure.AzureCDNBinaryManager.

That’s all! Try it out and let us know what you think.

The post Scale Your Blob Download Rate Using a CDN with Nuxeo LTS 2015 appeared first on Nuxeo.

.NET Framework is one of the most popular frameworks to develop for Microsoft Windows. Not only that, the adoption of ASP.NET as a server-side solution is increasing for the top 100k sites on the Internet, which makes .NET’s relevance hard to overlook. Keeping this in mind, developing a client using this framework for the Nuxeo Platform became one of our top priorities.

In this blog, I will discuss this client called the Nuxeo .NET Client in detail and show you how you can use it.

The Nuxeo .NET Client

The Nuxeo .NET Client is a cross-platform client library developed in C# for Nuxeo Automation and REST API. It targets two platforms:

• net45: the traditional .NET framework 4.5, which covers Desktop and ASP.NET apps, among others
• dnxcore50: for ASP.NET 5 apps using the open-source, multi-platform and recently-introduced .NET Core framework.

As such, the Nuxeo .NET Client is able to reach desktop apps for Microsoft Windows but also web applications developed with ASP.NET, running on Windows, Linux and Mac OS.

The Nuxeo .NET Client exposes a smooth interface that makes it easy to parameterize and send requests to an instance of the Nuxeo server. It takes care of marshalling and unmarshalling objects between native types and the JSON objects that are sent and received from the server.

Exploring the Nuxeo .NET Client

Now that we understand what it is and what it aims for, let’s play a little with it.

Installation

The Nuxeo .NET Client is available on NuGet. Assuming you are running Visual Studio 2013 or above, just open the Package Manager Console and type:

Install-Package NuxeoClient –Pre

and you are good to go!

If on the other hand, you are developing for ASP.NET 5 on .NET Core, you have to add the “NuxeoClient” dependency to your project.json file. It will look something like this:

{
   …,
   "dependencies": {
    …,
    "NuxeoClient": "1.0.0-*"
   },
  …
 }

Then, you should download the missing dependencies by running:

dnu restore

In this case, you should have .NET Core installed in your system, which will provide you the dnu command. Here are installation instructions for Linux, Mac OS and Windows.

Creating a Client

First of all, include a reference to the NuxeoClient namespaces in your .cs file:

using NuxeoClient;
using NuxeoClient.Wrappers;

Now, let’s create a new instance of NuxeoClient, assuming we have local Nuxeo server running with default authentication credentials:

Client client = new Client("http://localhost:8080/nuxeo/",
                        	new Authorization("Administrator",
                                          	"Administrator"));

Of course, the defaults can be omitted and we can end up with a much simpler instruction that does exactly the same:

Client client = new Client();

We can also add a default schema, which will be used on every request from now on:

client.AddDefaultSchema("dublincore");

We are now ready to issue requests to the server.

Common Operations

Here I will cover some of the most common operations that can be performed with the Nuxeo .NET Client.

CRUD: Create Document

You can create documents via REST API. Let’s create a folder in the root named “folder”:

Document folder = (Document)await
            	client.DocumentFromPath("/")
                  	.Post(new Document
                  	{
                        	Type = "Folder",
                        	Name = "folder",
                        	Properties = new Properties
                        	{
                            	{ "dc:title", "A Folder" }
                        	}
                  	});

This new “folder” will be our testbed henceforth.

CRUD: Update Document

Want a different title for that folder? Here’s how you can change it:

Document folder = (Document)await folder.Put(new Document
{
	    Properties = new Properties
	    {
    	   { "dc:title", "new title" }
	    }
});

CRUD: Create a Child Document

This is how you can create a file inside a folder:

Document file = (Document)await folder.Post(new Document
{
    Type = "File",
    Name = "TestFile1",
    Properties = new Properties { { "dc:title", "File 1" } }
});

Adapters: Getting Child Documents

Getting a document’s children can be achieved by using the @children adapter. This is how you can do it with the Nuxeo .NET Client:

Documents children = (Documents)await folder
                .SetAdapter(new ChildrenAdapter())
                .Get();

The Documents object encapsulates the respective JSON object with “entity-type”: “documents”. As such, it contains an Entries property, which represents a collection of Document objects. For example, checking for the number of children can be achieved with:

children.Entries.Count

From now on, every request to folder will be sent through the adapter. We can reset it with:

folder.SetAdapter(null);

Content Enrichers: Getting ACLs

Getting users’ permissions to read and write on a document can be conveniently achieved with enrichers:

Document folderWithAcls = (Document)await folder
                .AddContentEnricher("acls")
                .Get();

The ACLs can then be retrieved from the document’s ContextParameters with:

document.ContextParameters["acls"]

Batch Upload

The Nuxeo .NET Client provides an uploader that eases the task of uploading files. Let’s create a new instance of the Uploader which will upload files in chunks of 1kB:

Uploader uploader = client.Uploader()
                          .SetChunked(true)
                          .SetChunkSize(1024);

Now, let’s upload two files, “Test.txt” and “Puppy.docx”, to the server:

Entity result = await uploader.AddFile("Test.txt")
                              .AddFile("Puppy.docx")
                              .UploadFiles();

Batch Operation

We just uploaded two files to a batch, but they are not yet associated to any document. We would like to have them imported as children of our favorite folder. To do so, let’s perform a FileManager.Import operation on the whole batch, i.e., apply it to every document in it:

Documents documents = (Documents)await uploader
                .Operation("FileManager.Import")
                .SetContext("currentDocument", folder.Path)
                .Execute();

CRUD: Delete Document

Now that we are done with it, we won’t need our beloved folder anymore. Let’s delete it:

folder.Delete();

More on GitHub

In this post we just scratched the surface of what you can do with the Nuxeo .NET Client. Make sure you check its GitHub repository for more details, examples and documentation. The Nuxeo .NET Client is already being used by a sample application named NuxeoDotNetBrowser. It provides the basic functionality to browse documents in a Nuxeo server, perform CRUD operations and start workflows.

Nuxeo .NET Browser

The post Meet Nuxeo .NET Client: The Client Library for Nuxeo Automation and REST API appeared first on Nuxeo.

At Nuxeo, we’ve been using Docker for two years now, which is quite a long time in the Docker time scale! We built our Nuxeo.io infrastructure on top of it and are also using it in our CI chain.

I attended DockerCon EU 2015 in Barcelona and was very impressed by it. A lot of topics that were announced last year have now become a reality and they sure have caught my interest.  One such example is Docker Swarm, which is now able to schedule 50k containers in 30 minutes on AWS. Other examples are Docker Compose, now fully compliant with Swarm, and Docker Machine, which allows you to easily setup a Docker Swarm cluster in minutes.

Additionally, network and volume plugins have become first class citizens of Docker! They are now fully pluggable with solutions such as Weave or Flocker. This, l think, is a huge change in the Docker ecosystem because it allows us to build stateful isolated applications on top of containers.

In this post, I will share some feedback on my experiments with these new capabilities.

Building an Open/Close Container

As a developer, I often think about the open/closed principle. For Nuxeo Cloud, since we don’t have a generic dedicated Nuxeo image,I wrote a new and more open version of our Dockerfile which is now on Github. This allows you to configure some of the basic configuration properties. If it’s not enough there’s a way to add your own piece of nuxeo.conf.

It should be integrated into the official Docker library soon, so you will be able to  run a Nuxeo server on your Docker infrastructure:

$ docker run -d --name nuxeo nuxeo/nuxeo:LTS-2015

This is cool but it works with an embedded DB and an embedded Elasticsearch and is just there for testing purposes. We could add some links to other DB containers, but the setup is rather complex and relies on environment variables which never convinced me to use links in a general manner. Looks like I was not the only one since links are now deprecated in favor of using networking support.

Docker Compose and Networking Support

Docker Compose allows us to define an application as a set of containers by writing a simple YML configuration file. For instance, in the following configuration we will define a Nuxeo application with four containers: a Postgres DB, an Elasticsearch node, a Redis server and, of course, a Nuxeo server.

by running the command:

$ docker-compose --x-networking --x-network-driver=overlay up -d

It will start the four containers, create a dedicated network for that application and populate the /etc/hosts file with the addresses of the other services. So instead of links we now have some DNS name exposed in all our containers. It is then very easy to configure the Nuxeo Platform with the dedicated environment variables. The created network is private and no other container can see it if it’s not explicitly allowed to with Docker network commands.

The network support in Compose is still experimental, that’s why you have to manually enable it via the --x-networking command line option.

Compose also allows you to scale each part of your infrastructure. Of course, it has to be designed for scaling, but it’s very easy to do so. For instance, to add two other Elasticsearch nodes in our infrastructure, all we have to do is:

$ docker-compose --x-networking --x-network-driver=overlay scale elasticsearch=3

My Application in the Cloud

For now,  we are just deploying our application on a single node which means that even if we try to scale the small parts of our infrastructure, we will be limited by the host machine’s resources. That’s the type of problem that Docker Swarm can solve. Swarm is basically a tool that knows how to manage a cluster of Docker hosts and can schedule your containers across a cluster.

The clever thing about Swarm is that it exposes the same exact API as Docker, which means that launching a container across a cluster or launching it on a single host will share the exact same command. When you send a command to Swarm, it’s just the same as sending commands to Docker.

From a network point of view, it works just like it does on a single host. You can have the same private overlay network as in our first sample, but the overlay works across the hosts of your cluster. Just like Docker Compose talks to the Docker API, it can talk to Swarm too. It means that you can send the same Compose command to Swarm to launch your application. The containers will be distributed across the cluster, depending on the scheduling strategy. By default, it will spread the containers across the cluster. To sum up, here is a sample session to setup a cluster and deploy our application on it:

$ # Simplified version of the cluster's creation
$ docker-machine create --swarm --swarm-master swarm-master
$ docker-machine create --swarm  swarm-agent-00
$ docker-machine create --swarm  swarm-agent-01
$
$ eval $(docker-machine env --swarm swarm-master)
$ docker-compose --x-networking --x-network-driver=overlay up -d

The Docker Machine commands are simplified here. The full version is in the setup script of the sample Github project. In fact, it requires setting up a service discovery server (Consul, etcd or Zookeeper) and then making a reference to it. This servers allow Docker Swarm to correctly setup the network overlay and make things as smooth as possible.

So, we’ve been able to deploy our multi-tier application on a cluster of hosts with only a few commands. There is however a problem in this setup: if a host goes down or if we decide to move a container from one host to another, we may loose our data since we don’t use volume to persist them. Even if we had used volumes, they are linked to the host on which the container is running. We still have to find a way to deal with that in order to have a completely stateful multi-host application. I will be discussing that in my next blog. Stay tuned!

The post Fun with Nuxeo Cloud and Docker Tools After DockerCon appeared first on Nuxeo.

Chances are you are already familiar with Nuxeo Marketplace. It’s the Nuxeo ECM application store which offers plugins and packages that will enable you to easily add features to your Nuxeo application. One of the recent plugins we added in here is the customizable File System Exporter. This plugin enables you to export one chosen root structure of documents in the Nuxeo Platform to a specified File System repository on the server, and it can be customized too.

The Nuxeo File System Exporter

The Nuxeo File System Exporter can be installed following the instructions in the documentation.

In an earlier blog, we talked about the “File System Exporter” and you might be wondering what has changed since then that I am writing a new blog Well, we recently released the plugin as an official Marketplace plugin called the “Nuxeo FS Exporter”. In the previous blog, we discussed different examples of customizing the export by changing the default query and by using an extension point. Today, I will show you another example of customization by defining a new plugin.

In this new example, we want to be able to export only the main blob of a document (stored in the file:content field). Let’s get started!

How to Implement the Customized Export

Step 1: Install the Nuxeo FS Exporter plugin on your server. In Nuxeo Studio, add the JSON definition of the File System Exporter as explained in the documentation. You can now create an Automation Chain that uses this operation and is triggered by a User Action for instance.

Step 2: Create a new plugin.

Step 3: Manage dependencies. In the pom, you have to add a dependency to the fsexporter.

For example, for the version 7.10-SNAPSHOT this is what you need:

<dependency>
<groupId>org.nuxeo.io</groupId>
<artifactId>nuxeo-fsexporter</artifactId>
<version>7.10-SNAPSHOT</version>
</dependency>

Step 4: Write the Java class which will override the DefaultExporterPlugin.

Here’s an example of exporting only the main blob of a document:

package org.nuxeo.sample;

import java.io.File;
import java.io.IOException;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.nuxeo.ecm.core.api.Blob;
import org.nuxeo.ecm.core.api.CoreSession;
import org.nuxeo.ecm.core.api.DocumentModel;
import org.nuxeo.io.fsexporter.DefaultExporterPlugin;

/**
* Exports only file:content and overrides existing files, if any. Depends on nuxeo-fsexporter
*
* @since 7.10
*/
public class ExportOnlyMainBlob extends DefaultExporterPlugin {

public static final String DEFAULT_BLOB_XPATH = "file:content";
@Override
public File serialize(CoreSession session, DocumentModel docfrom, String fsPath) throws IOException {
File folder = null;
File newFolder = null;
folder = new File(fsPath);

// if target directory doesn't exist, create it
if (!folder.exists()) {
folder.mkdir();
}

if (docfrom.hasFacet("Folderish")) {
newFolder = new File(fsPath + "/" + docfrom.getName());
newFolder.mkdir();
}

Blob blob = null;
try {
blob = (Blob) docfrom.getPropertyValue(DEFAULT_BLOB_XPATH);
} catch (Exception e) {
// Ignore, we just have no blob here. Maybe we are handling a Folder with no "file" schema, it is not an
// error
blob = null;
}

if (blob != null) {
String fileName = blob.getFilename();
File target = new File(folder, fileName);
if (target.exists()) {
target.delete();
}
blob.transferTo(target);
}

if (newFolder != null) {
folder = newFolder;
}
return folder;
}
}

Step 5: You must now contribute this class in the OGSI/myNewExport.xml

<component name="org.nuxeo.sample.custom.fsexporter.onlymainblob">
<require>org.nuxeo.io.fsexporter.FSExporter</require>
<extension target="org.nuxeo.io.fsexporter.FSExporter" point="exportLogic">
<exportLogic class="org.nuxeo.sample.ExportOnlyMainBlob" />
</extension>
</component>		

That’s it!

Here’s an example of this export:

Customizable Nuxeo FS Exporter

Now you can create as many customized exports as you need!

The post Exploring the Customizable File System Exporter- a New Nuxeo Marketplace Plugin appeared first on Nuxeo.

Nuxeo UX Lab dashboard view

I recently wrote a blog to present the Nuxeo UX Lab, where I talked about our experiments with a new UI. This new UI enabled a quick access to documents and had improved collaboration tools to make it more user friendly.

At Nuxeo, we value your feedback and always try our best to meet and exceed your expectations. So we launched a UX survey for our users to know what matters the most and what they want to see in our next releases. We looked at the results and I am happy to say that you’ll love the Nuxeo LTS 2016 Roadmap!

I will briefly summarize the results of this survey and give you a sneak peek into the UI side of things that we are working on for the upcoming releases.

What Users Liked Most in the Nuxeo UX Lab

First of all, a warm thanks to everyone for exploring the Nuxeo UX Lab and giving us your feedback! Your feedback helps in boosting our energy to create better products and services.

Nuxeo UX Lab document view

The survey showed that our users liked the clean and simple UI. We kept that in mind as a guideline for our next development cycles.

The document view and the available actions in the sticky toolbar was another hit among the users. We are now working on further improving the current document view to make the actions even more accessible.

The users also loved the new Navigation and the reactiveness of the Nuxeo UX Lab and we are now putting in more effort to make the browsing between tabs, folders, tasks, and actions much smoother than before.

Most of all, everyone loved the side panel, which allows quick access to tasks and makes collaboration among teams very easy and pleasant. For the upcoming releases, our focus will be more on this side panel and the task notifier.

What Users Want to See in the Nuxeo Platform Web UI

The Nuxeo UX Lab has been created to experiment and test. Based on the survey, we found the main things our users want for the next Nuxeo Platform release:

• a foldable right panel to have shortcut access
• a better flow for tasks to treat them more efficiently
• an access to actions on table row to perform simple actions on the documents listed
• a simplified document view for more efficiency while working on it

We are currently working on the Nuxeo Roadmap to meet these requirements!

What’s Next: a Better UX for All Products and Services!

Besides the features mentioned above, we will also work on redesigning the Nuxeo Web Mobile application that will keep you up to date with your team’s timeline, notifications and tasks. In addition to these improvements and exciting projects on the Nuxeo Platform, we’ll put a great effort in improving the Nuxeo Studio experience by letting you customize the platform any way you want in a jiffy! And we won’t stop there! We also want to make the Nuxeo Online Services a key of success for your projects. So, we’ll work on that too.

2016 will be the year of an amazing product experience. Can’t wait to be in the future!

The post Nuxeo UX Labs: Our Survey, Findings, and Plan for the Future appeared first on Nuxeo.

For those of you who have previously enabled the storage of the Nuxeo Platform’s audit log in Elasticsearch, you may want to consider migrating your ‘audit’ index when you move to Nuxeo LTS 2015. Starting with Nuxeo LTS 2015 the ‘audit’ index has been renamed ‘nuxeo-audit’ (and, in fact, the name is configurable; see the upgrade notes).

There is no “out-of-the-box” way to copy or rename an index in Elasticsearch. In this example, we will use the Elasticsearch plugin ‘knapsack‘ to perform the migration.

Please note: the existing ‘audit’ index is not deleted after updating to Nuxeo LTS 2015, so any existing audit data will not be lost. The purpose of this migration is to ensure that the new index contains the existing audit data.

Install Knapsack

Go to https://github.com/jprante/elasticsearch-knapsack.

Locate the branch for your version of Elasticsearch. To get the Elasticsearch version, run ‘curl http://localhost:9200/’, adapting the URL for your server of course.  Here is some example output:

   {
      "status" : 200,
      "name" : "Book",
      "cluster_name" : "elasticsearch",
      "version" : {
        "number" : "1.5.2",
        "build_hash" : "62ff9868b4c8a0c45860bebb259e21980778ab1c",
        "build_timestamp" : "2015-04-27T09:21:06Z",
        "build_snapshot" : false,
       "lucene_version" : "4.10.4"
      },
      "tagline" : "You Know, for Search"
    }

Take note of the value in ‘version.number’.

Once the branch is located, check the readme for the installation command line. Copy the command line to install.

Note: you will need to locate the folder containing the Elasticsearch ‘bin’ folder, for example ‘/usr/share/elasticsearch’. Check the Elasticsearch Directory Layout documentation if you are not sure where to look.

For example, to install knapsack version 1.5.2.2 for Elasticsearch 1.5.x.

    cd /usr/share/elasticsearch
    ./bin/plugin -install knapsack -url http://xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-knapsack/1.5.2.2/elasticsearch-knapsack-1.5.2.2-plugin.zip

Once the plugin is installed, restart Elasticsearch. E.g.:

    /etc/init.d/elasticsearch restart

Prepare Nuxeo Upgrade

Prepare the Nuxeo application for the upgrade: create a backup, ensure you have the latest plugins, migrate the Studio project, etc. Be sure to stop the Nuxeo application during the index migration.

I recommend not upgrading Nuxeo until the index migration is complete because, for example, the update script automatically starts the server if using ‘apt’.

Export Existing ‘audit’ Index

Export the existing ‘audit’ index:    

    curl -XPOST 'localhost:9200/audit/_export?pretty&map=\{"audit":"nuxeo-audit"\}&path=/var/lib/elasticsearch/elasticsearch/nodes/0/nuxeo-audit.zip'

The ‘map’ parameter causes knapsack to export the index as if it were ‘nuxeo-audit’.

Note: ‘/var/lib/elasticsearch’ is the default location of ‘data’ for Elasticsearch, but you may need to adjust this path. Check the Elasticsearch Directory Layout documentation if you are not sure where to look. The point is to choose a location that the Elasticsearch server can write to, otherwise the export will fail.

You can check the status of the export with:

    curl -XPOST 'localhost:9200/_export/state?pretty'

Import Data to New Index

Import the data to the ‘nuxeo-audit’ index:

    curl -XPOST 'localhost:9200/nuxeo-audit/_import?pretty&path=/var/lib/elasticsearch/elasticsearch/nodes/0/nuxeo-audit.zip'

To be clear this will create the ‘nuxeo-audit’ index, with the same settings as ‘audit’, and import the data.

You can check the status of the import with:

    curl -XPOST 'localhost:9200/_import/state?pretty'

Verify Results

Check to see if the index was created:

    curl 'http://localhost:9200/_cat/indices?v'

If something looks wrong, delete the new index and try again:

    curl -XDELETE 'http://localhost:9200/nuxeo-audit/?pretty'

If all goes well, optionally delete the old ‘audit’ index.

    curl -XDELETE 'http://localhost:9200/audit/?pretty'

Complete Nuxeo Upgrade

Complete the Nuxeo upgrade. When the server starts up the new ‘nuxeo-audit’ index will be used.

The post Migrating the Nuxeo ‘audit’ Index in Elasticsearch appeared first on Nuxeo.

Today I will talk about a simple topic, which is more of a tip actually!

Did you know you can directly edit the files attached to a document with your usual software? It’s very easy! Thanks to Nuxeo Drive, you can use the software you have locally assigned to the file type for editing. So, it’s a very straightforward integration with tools, such as Photoshop, MS Office, etc.

With the recently added features, you can edit a file directly from the document listing, which means you don’t need to actually go to the document itself. Also the document will be locked so nobody else can edit the same document at the same time as you. Of course it gets unlocked when you are done.

Check out this quick video to see it in action:

The post Quickly and Safely Edit Files in Nuxeo appeared first on Nuxeo.

The Nuxeo REST API provides a wealth of information about documents in the repository, and it’s awesome that the Nuxeo Elements and Data Visualization Polymer components allow us to leverage the REST API to build amazing, modern Web applications.

One of the challenges I ran into in terms of creating an effective user experience had to do with dealing with file types. I wanted to build a dashboard chart to display the distribution of different file types within my application. The Nuxeo REST API supplies the MIME type for a given blob when accessing the documents, but for user-friendliness it is generally better to display the file extension or even a “friendly name” rather than the MIME type. In the following screenshots the same data is being displayed but the latter is certainly easier to understand:

So I needed a way to map a MIME type to something more user-friendly.

There are plenty of JavaScript implementations to map MIME types to something else but I found nothing specifically for Polymer. In Polymer applications you are meant to deal with Web components and not random JS code, so I thought it might be nice to build a simple Polymer element to tackle this task. Enter `nuxeo-util-mimetypes`. This is a Polymer element that can provide the file extension or a “friendly name” given a MIME type.

Here is an example of fetching Nuxeo repository data and aggregating on the MIME type of the `file:content` blob using the `nuxeo-repository-data` element:

    <nuxeo-repository-data ecm-primary-type="File"
               grouped-by="file:content.mime-type" 
               data="{{typeCount}}">
    </nuxeo-repository-data>

The results of this request might return an object like this for Word documents:

    {
     "key": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
     "doc_count": 26
    }

The MIME type “application/vnd.openxmlformats-officedocument.wordprocessingml.document” is quite useless for most end-user interfaces. Instead I can use the `nuxeo-util-mimetypes` element to get the friendly name. Here is an example using JavaScript from within another Polymer element:

    var mimeType = ‘application/vnd.openxmlformats-officedocument.wordprocessingml.document’;
    var mimeTypeLookup = document.createElement('nuxeo-util-mimetypes');
    mimeTypeLookup.mimeType = mimeType;
    mimeTypeLookup.friendly = true;
    var friendlyName = mimeTypeLookup.query();

The `nuxeo-util-mimetypes` repository contains documentation and a working example.

To use the `nuxeo-util-mimetypes` element in your Polymer application just install it via bower:

    bower install --save nuxeo-sandbox/nuxeo-util-mimetypes

You can find a more fleshed out example using `nuxeo-util-mimetypes` in the `nuxeo-dam-dashboard` element.

Note: the MIME type data provided with `nuxeo-util-mimetypes` comes mainly from:

• http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/conf/mime.types?view=markup
• https://developers.google.com/drive/web/mime-types

There are a few “friendly names” already implemented but your pull requests with more additions are most welcome!

The post Making Friendly Polymer Apps appeared first on Nuxeo.

If you are an Administrator of the Nuxeo Platform, getting a quick overview of where the Documents are located in the repository might be one of your most common tasks. The good news is that this can be done easily! You can now leverage the nested aggregations provided by the Elasticsearch index used in the Nuxeo Platform to produce a sunburst pie chart representing the disk usage of your Nuxeo deployment within Kibana. I will walk you through the configuration process and the steps on how to achieve this.

Configuration

Requirements:

• Nuxeo platform 6.0 or greater, Nuxeo LTS 2015
• Elasticsearch 1.4.4 – 1.7
• Kibana 4.1.x

To be able to build the above chart, we need to introduce an Elasticsearch transform in the mapping file. The Elasticsearch mapping file is located in the $NUXEO_HOME/templates/common-base/nxserver/config/Elasticsearch-config.xml.nxftl.

Add the following transform based on a groovy script:

 ...
    "transform": {
        "lang": "groovy",
        "script": "def splitPath = [];splitPath = ctx._source['ecm:path'].split('/'); ctx._source['pathDepth'] =splitPath.length; for (i = 1; splitPath.length > i; i++) { ctx._source['pathLevel' + i] =splitPath[i] }"
    },
    "properties" {
 ....

Note that the transform definition goes at the same level as the mapping properties. Let’s take a closer look at the groovy script.

def splitPath = [];
splitPath = ctx._source['ecm:path'].split('/');
ctx._source['pathDepth'] = splitPath.length;
for (i = 1; splitPath.length > i; i++) {
   ctx._source['pathLevel' + i] =splitPath[i];
}

The script is going to be invoked each time a Nuxeo Document is indexed in Elasticsearch. It will extract the ecm:path string property (e.g. /default-domain/worspaces/aWorspace/aFile) from the source of the Document. This path is then split between the ‘/’ character in order to index each part of the path in a dedicated field called pathLevel1, pathLevel2, pathLevel3, etc. depending of the depth of the Document in the hierarchy.

Note that Groovy dynamic scripting is off by default from Elasticsearch v1.4.3 and you need to enable it from your $ELASTIC_HOME/config/Elasticsearch.yml:

script.groovy.sandbox.enabled: true

Once this is done, you must re-index your documents into Elasticsearch so that the transform is applied and pathLevel1, pathLevel2, pathLevel3, etc. fields are valued. To do so:

• Go to the Admin Center of your Nuxeo deployment
• Click on the Elasticsearch menu
• Go to the Admin tab
• Click Re-index repository

That’s it! Let’s now build a chart on top of these new fields.

Build the Chart with Kibana

We will now see how to design the chart in Kibana in a few steps:

1. Go to the Visualize tab
2. Create a Pie Chart by clicking:
   
   
3. Click From a new search
4. The chart design assistant appears on the left-hand side panel:
   
   
   
   Click Split Slices
5. Setup the nested aggregations on the different path level fields created by the transform groovy script:
   
   
   
   At the first level, select Terms aggregation type on the pathLevel1 field. In the Size input, enter 0. It will force Elasticsearch to return exhaustive counts.
6. Click Add sub-buckets and repeat step 5 as many times as you want/need depending on the maximum depth of your hierarchy. Each time, select a deeper pathLevelx field (e.g. pathLevel2, pathLevel3, etc.).
7. Click the Apply changes button:
   
   
8. Enjoy your sunburst chart:

Troubleshooting

Nothing appears? That can happen for one of these reasons:

By default, Kibana fetches entries created in the last 15 minutes. In the top right corner, click on:

and select an appropriate time range, for example Last 5 years.

If you can’t see pathLevel1, pathLevel2, etc. in the field selector when defining your buckets in step 5 and 6, you may need to Reload field list. Go to the Kibana Settings menu and click the Orange button for the Nuxeo index.

What About File Size?

The previous steps guide you to build a chart that shows how many Documents you have in each folder. In order to build a proper Disk Usage chart, you need to customize the nested aggregations a bit further.

Luckily, the Nuxeo Platform computes and stores the size of each Document (more exactly the size of their attached binaries) in the common:size Document property.

To create a Disk Usage chart, redo the previous steps and instead of selecting a Count aggregation in the metrics definition, select Sum on the common:size field.

By the way, you may want to restrict the size computation for particular Documents. In the above screenshot, you can see that I added the following query clause:

-ecm.mixInType.HiddenInNavigation -ecm.currentLifeCycleState.deleted

to exclude the hidden Documents (typically system documents) and the deleted Documents (i.e. located in the trash bin).

Here’s the final result:

The post Browse your Document Distribution with Kibana appeared first on Nuxeo.

Elasticsearch is the cornerstone of the Nuxeo Platform’s search capabilities. We made a deep integration with Elasticsearch by adding a higher level of configuration within the Nuxeo Platform for features such as reindexing, index optimization, passthrough, etc., which helps us provide the best search experience to our users.

Recently, we did some work on Elasticsearch synonyms configuration. Our goal was to make it easier for the platform Administrator to enrich the list of synonyms without having to access the Nuxeo server where the synonyms.txt file is stored. In this blog, I will give you an idea of the work we did and share the result.

Configuring Synonyms

We created 3 simple operations:

• ReadSynonyms: To read the content of the synonyms and display it on the UI
• WriteSynonyms: To write the updated synonyms back into the text file
• Reindex: To trigger reindexing Elasticsearch after the synonyms are updated

We then bootstrapped a Polymer project based on this template.

Next a simple element called <nuxeo-synonyms> was created. This element contains a textarea (<paper-textarea>) and a button (<paper-button>). At the initialization of the element, all we have to do is make a call to the ReadSynonyms operation and initialize the textarea. That’s it! Also, you might have guessed that when you press ‘Save’, the WriteSynonyms operation is called that saves the new synonyms in the text file.

Each line represents a group of synonyms, and a synonym can be a word or a group of word.

Now, depending on your configuration you’ll have multiple options:

1. Triggering the auto reindex every time you click on Save (could come in handy for a quick demo or even in production if you don’t have millions of documents to reindex)
2. Reindexing your repository manually once you are done with your modifications
3. Using the Elasticsearch hint feature that enables you to use synonyms without reindexing (This feature is not available right now, but once it is available I will write a blog on how to configure it. So stay tuned!).

To enable the auto-reindex, all you need to do is add org.nuxeo.synonyms.autoreindex=true in nuxeo.conf

Now every time you search for a word or it’s synonym you’ll get the same results!

The github repository is here. Currently, the project is in sandbox mode but it can be used as is. So, go ahead and explore it! We will soon roll it out as a supported Nuxeo Package.

The post Elasticsearch Synonyms Configuration UI within Nuxeo appeared first on Nuxeo.

A few months ago, the Nuxeo Platform LTS 2015 introduced file permissions as a way to restrict access to files in a document.  One of the most obvious use case of that feature is Digital Asset Management as the Nuxeo Platform automatically computes several conversions of picture and video assets for different resolutions and formats.

A Picture Asset

Here is an example. Let’s say that for picture assets, the main file is an Adobe Photoshop file whose access should be restricted only to the creative team while the JPEG conversions should be made available to any consumer user who has access to the asset. File permissions enable us to do that. Now let’s take it one step further and say that all those users should still be able to synchronize assets on their desktop. The creative team user can synchronize the Photoshop files while other users can synchronize the JPEG conversions.

Since Nuxeo Drive synchronises the main file by default, it’s not going to work for users who don’t have the permission to download that file. However, the Nuxeo Platform provides an easy way to implement this use case without modifying Nuxeo Drive. The only thing to do is to override the default BlobHolderFactory class that Nuxeo Drive relies on and replace it by an implementation that includes our business logic.

public class CustomPictureBlobHolderFactory implements BlobHolderFactory {
        BlobHolder blobHolder;
        if (doc.hasFacet("Picture")) {
            NuxeoPrincipal principal = (NuxeoPrincipal) doc.getCoreSession().getPrincipal();
            if (!principal.isMemberOf("Creatives")) {
                blobHolder = new PictureBlobHolder(doc, "picture:views/3/content");
            } else {
                blobHolder = new PictureBlobHolder(doc, "file:content");
            }
        } else {
            blobHolder = null;
        }
        return blobHolder;
}

That’s it. Nuxeo Drive now synchronizes the main file for the creatives and the JPEG conversions for other users.

Asset Folder

PSD files in the local folder of a user who is a member of the Creatives user group

JPEG files in the local folder of other users

The complete source is available in our nuxeo-sandbox github repository. Try it out and let us know what you think!

The post Nuxeo Drive and File Permissions appeared first on Nuxeo.

If you are looking for information about your Nuxeo Platform, such as workflow performance, the most frequently used search terms, the number of documents created last month, or who created the most documents in your application, then you are reading the right blog!

Starting from the LTS 2015 release, the Nuxeo Platform offers a data visualization toolkit, based on HTML5 and Web Components. This toolkit will help you visualize content data and analyze activities, such as workflows, search requests, etc.

A set of widgets is available for configuring graphical dashboards that will allow you to monitor your data and workflows.

This feature leverages Elasticsearch distributed computation capabilities that allows you to aggregate, compute, and then visualize any data stored in the Nuxeo Platform.

What is Available Out-of-the-Box

The Nuxeo Platform contains sample dashboards built using available widgets. For example:

• Pie charts and bar diagrams for distribution of metadata values
• Tables for nested aggregate computation, such as average value of a given property distributed among terms of another property of the document

Here’s what you can find by default.

Measuring Workflow Performance:

Make sure that the add-on “Nuxeo Review Workflows Dashboards” is installed from the Marketplace ( it can be installed during the Nuxeo Platform startup wizard or later from the Admin page).

As an administrator, you can access the list of available workflows in the Admin tab > Workflow section. Select the one you want and by default you will be on the “Analytics” tab of the selected workflow.

You can directly see the following information about a given workflow:

• Average duration
• Distribution of the initiators
• Average time spent per user on each node of the workflow
• List of actions per user

You can see the analytics for the default embedded workflow “Parallel Document Review” below. Note that at the top you can select the date ranges to refine your information.

“Parallel document review” workflow analytics

Repository and Search Analytics:

As an administrator, you can access the “Repository Analytics” and “Search Analytics” in the Admin Tab > Activity Section.

You can find the following information about the Repository Analytics:

• Number of documents
• Distribution of documents types
• Top 10 creators
• Graphics with number of created and modified documents between two date ranges

Repository Analytics

Regarding Search Analytics, the following information is provided:

• Distribution of search type used
• Bar diagram with number of calls per hour
• Tables with the most used search terms, number of results, number of pages displayed and filters used

Search Analytics

Customize your Dashboards

You can quickly:

• Create advanced analytics dashboard on content/data or performance (workflow, activity)
• Aggregate any query on one or multiple axes (time, bucket, terms)
• Render using advanced charting elements, leveraging any JS charting library

Check out this documentation to learn more about building your own dashboards.

The post Monitor your Data and Workflows with Data Visualization and Analytics in Nuxeo appeared first on Nuxeo.