FOSSology Project Logo FOSSology
Advancing open source analysis and development
 

Notes for tutorial section

A module is some functionality that is added to FOSSology. To get started extending FOSSology, let's go through a series of working examples:

  1. Hello World UI plugin (mary)
  2. UI plugin - more detailed description of FO-plugin and referring the reader to docs for additional functionality
  3. Advanced UI plugin (mary)
  4. Explanation of API (rando)
  5. Add agent to Advanced UI plugin (mary)
  6. Explanation of wc agent (mary)
  7. Explanation of SQL Agent and Shell agent (mary)
  8. Explanation of schema (bobg)
  9. Review FOSSology Tutorials (mike)

Getting information out of FOSSology

  1. UI method (Using the FOSSology UI)
    • Use the installed FOSSology plugins
    • Create a custom FOSSology plugin
    • Non-FO UI query - E.g. external website (Eddy?)
    • Hybrid - FO plugin that points to a non-FO UI (FO plugin that queries external website)
  2. Non-UI method
    • Direct database query
    • Using wget to download FOSSology info via the repo's URL

Getting data into FOSSology

  1. UI method
    • Upload from URL
    • Upload from Server
    • Upload from File
  2. Use an installed FOSSology plugin
  3. Create a custom FOSSology plugin (may require modification to scheduler.conf)
  4. Adhoc agent
    • Use scheduler
      • sqlagent, shell agent (requires modification to scheduler.conf)
    • Without scheduler
      • Direct db access
      • wget POST
  5. cp2foss - command line upload utility

Use an agent when...

  1. It needs to run as a “batch” process (takes a long time)

Specific Use Cases

  1. Lawyer - Show me all license categories, as defined by me.
  2. Lawyer - Show license change history for a given package.
  3. OSS researcher
  4. Developer
  5. Package Maintainer
  6. Redistributor/Integrator (E.g. HP distributing RedHat, Tivo including opensource firmware in product)

Example of a plugin to display a sorted table containing a license histogram of uploads

Note: This could also be a single long query but, we will make it a plugin for the purpose of example.

v1: I am a lawyer who wants a license histogram of all debian uploads. (actual code for this plugin is in trunk/devel/tutorials/ui-plugins/ui-debian-lics_v1.php)

  • Query the db for all debian uploads; get the corresponding upload_pk (unique identifier for each upload) & uploadtree_pk (a tree that represents the directory structure of the upload). This requires joining two tables on the upload key. A direct query would look like this:
 fossology=> SELECT uploadtree_pk, upload_pk, upload_filename FROM upload INNER JOIN uploadtree ON upload_fk=upload_pk AND parent IS NULL WHERE upload_filename LIKE '%debian%';
  
 uploadtree_pk | upload_pk | upload_filename
---------------+-----------+----------------------------------------------------------------------------------------------------
          3460 |         8 | http://debian.osuosl.org/debian-cdimage/4.0_r5/source/iso-dvd/debian-update-4.0r5-source-DVD-1.iso
(1 row)

In my case, I only have one debian upload.

  • In my php plugin, the query above is stored in the variable $SQL and passed to the db for execution. The results of the query are passed back in the array $Results.
 $Results = $DB->Action($SQL);
  • Next, loop thru the results and get every license for every file in this uploadtree. For this, we call the common function LicenseGetAll ( found in trunk/ui/common/common-license.php) and it returns the list of licenses in the array $Lics.
 foreach($Results as $Row)
    {
    if (empty($Row['upload_pk'])) { continue; }
    else { LicenseGetAll($Row[uploadtree_pk], $Lics); }
    ...
    }
  • Sort the results and output to a table:
 arsort($Lics);
 foreach($Lics as $key => $value)
   <format table and output to UI>   (see the code for details)

v2: Using the previous example, make all the licenses “clickable” such that when they are clicked, they display all files with that license. This example will illustrate how to call a plugin from a plugin.

We need to check for the existence of the plugin (search_file_by_license) we wish to use. Use the function plugin_find_id found in the common code.

  if (plugin_find_id('search_file_by_license') >= 0)

Now, we simply make the license text a link that calls the plugin to locate all files with the given license. Change this line of code:

 "<tr><td>$key<td align='right'>$value\n";

To this:

 "<tr><td><a href='/repo/?mod=search_file_by_license&item=$Row[uploadtree_pk]&lic=" . urlencode($key) . "'>$key</a><td align='right'>$value\n"

v3: Use V2 but, prompt for the search string (i.e. remove hardcoded %debian%)

To add a prompt for the search string, we'll borrow a code snippet from an existing plugin, search-file.php. This code uses a function (GetParm) from the common code library to retrieve the search string safely and check it's data type.

      $Filename = GetParm("filename",PARM_STRING);
      $Uri = preg_replace("/&filename=[^&]*/","",Traceback());
 
      /* Prompt for the string to search for */
      $V .= "<form action='$Uri' method='POST'>\n";
      $V .= "<ul>\n";
      $V .= "<li>Enter the string to search for:<P>";
      $V .= "<INPUT type='text' name='filename' size='40' value='" . htmlentities ($Filename) . "'>\n";
      $V .= "</ul>\n";
      $V .= "<input type='submit' value='Search!'>\n";
      $V .= "</form>\n";

Check to see if we have string to search for and insert it into the SQL statement:

      if (!empty($Filename))
        {
        /* Get uploadtree_pk's for all debian uploads */
      $SQL = "SELECT uploadtree_pk, upload_pk, upload_filename FROM upload INNER JOIN uploadtree ON upload_fk=upload_pk AND parent IS NULL WHERE upload_filename LIKE '%$Filename%';";

The remaining code is as before in the previous version.

Example of a "package agent" that looks for packages (rpms, debs, ...) and flags them in the DB

More info on how to create an agent

 
task/how_to_write_agents_plugins_tutorial_with_examples.txt · Last modified: 2009/02/27 14:55 by laser

Copyright (C) 2007-2009 Hewlett-Packard Development Company, L.P.
FOSSology Project documentation is licensed under the GNU Free Documentation License Version 1.2
Recent changes RSS feed Valid XHTML 1.0 Valid CSS3 Driven by DokuWiki