How to Add a License Template or License Phrase and Re-Analyze Licenses

The license templates are a set of known licenses used for the analysis. When any part of the known license (called a raw template) match, then the license is marked as a match. However, new or unknown licenses may not match well. In some cases, you may want to add in your own license.

Currently, adding licenses is not user-friendly. It requires re-running the build and modifying the database.

There are two places where you can put your raw license template:

1. Go into the Raw license directory in the build tree:

   cd trunk/fossology/agents/foss_license_agent/Licenses/Raw/

   This directory contains all of the “Raw” license templates. For organization, templates are created directories. You can either put your raw license in one of these directories, or make your own.

2. When you run “make install”, this entire tree is copied as-is into the install directory: trunk/fossology/install/usr/local/share/fossology/agents/licenses/. When you run “sudo make install”, the following steps happen:
   • The tree is installed in /usr/local/share/fossology/agents/licenses/
   • Filter_License is used on every file to generate the License.bsam cache and to install the license meta data in the database. This happens when you see the ”…” during the install.

Alternately:

1. Place your file under /usr/local/share/fossology/agents/licenses/.
2. Run 'sudo fossinit'. The ”…” sequence indicates that the licenses are being installed.

NOTE: When doing this alternate install, the migration command 'fo-cleanold' will remove the installed license directory and the templates.

After installing the license, use the UI to set any canonical names for the new license: Organize → License → Manage Terms. Your new license will be installed under its own canonical name. If it should be part of another license group (e.g., it is another variant of the GPLv2 license reference), then delete the current canonical name and add the license to a different group. Alternately, you can change the canonical name or add license terms for the new license.

There are some general guidelines when selecting raw license text:

• Raw licenses. You should not need to edit any of the actual license text. The “raw” directory contains unmodified licenses.
• Separate parts. Many licenses contain distinct parts that can and should be separated. For example, a preface or history behind the license does not alter the meaning of the license. Removing these sections speeds the comparison process time. Similarly, appendices should either be removed or separated into their own license files.
• Short licenses. Many full licenses contain sample text for a minimal (short) license declaration. For example, “Files using this license should contain the following paragraph…” These paragraphs should be extracted into separate files so that they match the short license.
• Virtually identical licenses. Try to stay away from “virtually identical” licenses. Many project use the same license but change the copyright name. It is my understanding as a non-lawyer that different names do not change the meaning of the license. The only thing they change is the copyright owner – the owner is the person who has the option to alter the license. As a result, two variations of the BSD license that only differ by the copyright holder's name are both identified as the BSD license. In general, if your company likes the BSD license, then they will probably like the same license held by a different copyright holder. However, it is up to your legal department to determine whether similar licenses have similar desirability.

One sentence license phrases (1SL) are phrases commonly associated with licenses. When no license template is found, 1SL phrases are displayed.

Currently, all 1SL templates are hard-coded into the Filter_License agent. The file is: trunk/fossology/agents/foss_license_agent/Filter_License/1sl.c

At the very beginning of the code is a global array named “List1SL”. The array ends with NULL,NULL. If you want to add in your own one-sentence-license (1SL), then just add the pattern before the NULL's.

Each 1SL array entry has two parts:

1. A name. (I use “1SL: %s”)
2. A regular expression that isn't quite regular. (See wordregex.c for default.) The expression terms are as follows:
   • ”%” skip 0 or more words (this expression is computationally expensive)
   • ”%5” skip up to 5 words
   • “string” match string
   • “^string” do NOT match string
   • “string*” match word beginning with string
   • “*string” match word ending with string
   • “*string*” match word containing with string
   • “*^string*” do NOT match word containing with string
   • “*” match exactly ONE entire word (same as %1)
   • “string1|string1” match string1 or string2
   • ”< … >” place matches in return string
   • “\” quote next character (only for start of match)
3. All words are case-insensitive.

As an example one-sentence license phrase:

< * * proprietary % > *.*|*,*|*;*|*:*|*$*|*(*|*)*|*{*|*}*

This expression looks for any set of words containing the word “proprietary”. It returns 2 words before “proprietary” and then any number of words until it finds an end-of-phrase character (period, comma, semi-colon, etc.). This will match phrases like:

This software contains proprietary source code.

(The expression returns “software contains proprietary source code”)

This gets around proprietary Microsoft APIs.

(A real phrase from some open source projects.)

Maybe this should be proprietary?

(Returns “should be proprietary”)

After changing this file, use “make” in the Filter_License/ directory to make sure that it builds. Then use “make” from trunk/fossology/ to build the code and “make install” to install it.

Changes to the license templates or 1SL system are not currently applied to previous analysis results. As a result, files that are already processed will not be reprocessed with the new license.

New uploads should be reanalyzed through the UI.

1. Click “Browse” at the top menu. Navigate the tree until you find the upload to analyze.
2. Under the upload's description is a link to “Reschedule license analysis”. This will schedule the upload for a re-analysis.

Alternately, you can reschedule jobs using the command-line interface:

1. The following command lists each upload: /usr/local/lib/fossology/agents/delagent -l
2. Delete the licenses associated with a specific upload. For example, to remove licenses from upload number 3 (number comes from -l), use: /usr/local/lib/fossology/agents/delagent -L 3
3. Use the 'fossjobs' command to reschedule license analysis: fossjobs -a agent_license -U 3. To see the list of jobs, use fossjobs -u. You can also specify analysis for all jobs (that do not have analysis) by using fossjobs -a agent_license -U ALL.