**CDIGS Management Report (Feb 15 -- Mar 15, 2007)**

**Key Accomplishments:**

**Released Globus Toolkit 4.0.4 on Feb 23.**\
Toolkit point releases are the easiest way for facilities to get
bug fixes and improvements.  This release contains a number
of improvements, detailed in the release notes at
<http://www.globus.org/toolkit/releasenotes/4.0.4/> .  These include
platform-specific build improvements for sites like the Teragrid as well
as general performance and scalability enhancements. WS-Gram for example
was updated to manage the OSG use case of handling a single user
submitting 1000 jobs.) Releases also include documentation and release
notes for all the updated components.

**Began GT 4.0.5 feature merge (targeted for April)**\
This process is important for synchronizing our major user communities
on a single codebase.  When this feature merge is done, we will reduce
the number of patches between the GT releases used by  Teragrid, OSG,
and other VDT-using communities.  In addition, it will get
community-driven features like GRAM auditing out in a directly 
supported release.

**Accepted four additional incubator projects, thereby increasing the
functionality offered within Globus and extending the user community.**

**Alpha release of WS GRAM with JSDL support**\
This was initially made available to the AHE project team -
http://wiki.ngs.ac.uk/index.php?title=Application_Hosting_Environment\
This will be available for all users in the upcoming GT 4.1.1
development release\
In order for WS GRAM to maintain and grow it's user base, it is
important to support emerging standards like JSDL.

**Completed a competitor and customer analysis report for Java WS
Core.** Started work on evaluating some similar tools (Muse, Axis 2) to
evaluate opportunities for collaboration.

**UK requirements gathering workshop** (Jointly funded by UK eScience
Program and JISC) <http://www.nesc.ac.uk/esi/events/743/> Draft report
available here <http://www.nesc.ac.uk/talks/743/draft_report.pdf>
Requirements gathering is fundamental to understanding where our effort
should be spent. The presented report as the result of over 30
interviews in the UK, and the workshop gathered feedback from a large
community.

**Ongoing work on the RLS Java client:** This roadmap item is an
important one for several of our application communities, including ESG
and the Pegasus team that runs workflows for LIGO, SCEC and many other
projects. The native Java client will overcome existing difficulties on
64-bit platforms. The development of the Java client has required
significant reverse engineering of the protocol implemented by the
original developer of RLS.

**Helped external groups to develop modules for GridFTP**

\- ESG on creating an OpenDap Data Storage Interface for GridFTP

\- caBig on creating an authorization module for GridFTP

> \- NorduGrid on adding some features of version 2 of GridFTP protocol
> in the GridFTP clients.

**Created specification for delayed passive** (that was missing in the
GridFTP protocol specification), an optional mode of operation for the
passive data channel commands of GridFTP.

**Developed a faster way of creating new doc directories** - moving
forward, we will be releasing all docs on a point basis (including
stable releases). In other words, instead of having all docs for 4.0.x
reside in one directory, there will be a directory for each point
release: 4.2.0, 4.2.1, etc

**MDS Trigger**

-   Trigger how to work is almost complete, waiting only on a patch
    (\*1) <http://bugzilla.globus.org/globus/show_bug.cgi?id=3972> Added
    documentation to make it much simpler to use the Trigger service,
    and explain how users can set up Triggers on their own.

-   Ongoing work towards 4.0.5 release (\*1,3)
    <http://bugzilla.globus.org/globus/show_bug.cgi?id=5099> This
    includes significant new functionality originally developed for
    TeraGrid in terms of information providers, and ease of developing
    new information providers, as well as scalability enhancements.

**Metrics**

-   Dealt with operational issues with the usage data database
    service (disk filling up) caused by extra analysis activities.

-   Assisted TG reconfigure WS containers across TeraGrid to send data
    to both Globus and TeraGrid listener services. TeraGrid\'s GT4 WS
    containers (which provide WS GRAM and RFT services) were
    reconfigured to send usage reports to both the Globus listener and
    to a new TeraGrid listener service.  This provides TeraGrid
    with TeraGrid-specific usage data for WS GRAM, RFT, and the WS
    Container itself. This data will be useful in TeraGrid for many
    things, including monitoring the growing use of these services. 
    Data collected by the Globus listener was used on a one-time basis
    for the TeraGrid annual report, and this data was viewed as quite
    important by the TeraGrid review panel. We will have considerably
    more data for use throughout the year and in subsequent annual
    reviews.

-   DRS team began adding usage reporting capability to DRS and a packet
    handler to the listener service.

-   Automated Globus Toolkit download statistics generation,
    so collecting this data for our semi-monthly Globus Usage reports is
    no longer a manual process.  (This is an internal process
    improvement, saving future effort in generating usage reports.)

**2. Technology, process, and performance improvements**

**Added a new globusrun option to delegate a full proxy in WS Gram.**
This is useful for clients submitting one job and then allowing that job
to submit another.

**Core Common Utils**

-   Added tool (globus-hostname-lookup) to troubleshoot problems with
    hostname resolution and expected DNs (bug 3149)

-   Added automatic detection of the number of CPUs available, and set
    the number of threads to take advantage of this.

**Security:**

-   Factor out the authorization engine in Java WS Core to ensure that
    the framework can be used independently of the core ws-framework.
    Benefits: Some user communities have expressed interest in using the
    Framework for providing attribute-based authorization at the
    application level and extracting this as a separate module will
    allow for this. Also, the Java interfaces for PDP and PIP can be
    standardized. (4935)

-   Updated the LoginModule to use UsernameAuthorization.

-   Support for password digest as a part of the username/password
    security

-   Updated C delegation client utilities to support proxy refresh
    operations.

-   Created a new command-line client program and relevant documentation
    for the delegation service.

**GridFTP:**

-   Enabled dynamic loading of ftp client library plugins with
    globus-url-copy.\
    Cleaned up ACL module plugin system, made modules dynamically
    loadable\
    and added example module.

-   Added audit callout to ACL system for better server instrumentation.

**RLS/DRS**

-   Ongoing design discussions regarding higher-level data services,
    which will extend the existing Data Replication Service (work also
    funded by CEDPS).

-   Shishir Bharathi (PhD student) completed implementation of a
    peer-to-peer version of the GT4 WS-RLS.

```{=html}
<!-- -->
```
-   4926: Delegation utility does not use EPRs

-   4856: cannot find input file: programs/grid-cert-renew.in

-   4896: globus_gsi_cert_utils_make_time() timezone handling fixed.

-   4979: Escaped double quotes not handled correctly

**Security (automated tests)**

-   Enhanced tests for parameter based authorization (Bug 4893)

-   Wrote Junit tests for the security module to check for valid SOAP
    parameters. (Bug 4893)

-   Tested the Username Authorization PDP, which is part of the PDP
    framework with a sample Login Module.(Bug 4837)

-   Created tests for UsernameAuthorization by using the Login Module,
    Configuration Class and a TestService and Testclient.

**Testing infrastructure**

-   Investigated an easier setup methodology for testing purposes and
    developed a custom Configuration class to setup the environment
    programmatically.

**dev.Globus**

-   Additional work in defining the Incubator Evaluation Process

-   Preparation for an Incubator process review in March

**MDS**

-   Ongoing work towards 4.0.5 release (\*1, 3)
    <http://bugzilla.globus.org/globus/show_bug.cgi?id=5099>\
    This includes significant new functionality originally developed for
    TeraGrid in terms of information providers, and ease of developing
    new information providers, as well as scalability enhancements.

-   Updated framework for tracking all MDS4 project work in bugzilla
    (\*1,4)

    -   <http://snipurl.com/1cyvx>

-   Completed GridFTP performance information provider (\*1, \*2)

    -   <http://bugzilla.globus.org/globus/show_bug.cgi?id=3977>

-   Ongoing support work for TG deployment of MDS as information source
    for portal (\*1, \*3)

    -   <http://bugzilla.globus.org/globus/show_bug.cgi?id=5057>

    -   Also work developing information providers specific to PSC
        machines and Purdue resources for TG portal (\*1, \*3)

        -   <http://bugzilla.globus.org/globus/show_bug.cgi?id=5058>

        -   <http://bugzilla.globus.org/globus/show_bug.cgi?id=5086>

-   Ongoing work for registry for TeraGrid CTSS (\*3)

    -   <http://bugzilla.globus.org/globus/show_bug.cgi?id=5070>

-   Ongoing work improving MDS4 test suite, as part of component handoff
    work (\*1)

    -   <http://bugzilla.globus.org/globus/show_bug.cgi?id=5065>

-   Began trigger update work, primarily motivated by ESG, to improve
    scalability and state management (\*1, \*4)

    -   <http://bugzilla.globus.org/globus/show_bug.cgi?id=5067>

-   Continued work on prototype web interface to triggers to allow
    easier deployment and registration by users(\*1, \*4)

    -   <http://bugzilla.globus.org/globus/show_bug.cgi?id=5068>

-   Completed work to suppress the publication of aggregator
    configuration elements in aggregator service group registry entries,
    originally in response to ChinaGrid (\*1)

    -   <http://bugzilla.globus.org/globus/show_bug.cgi?id=5059>

-   Completed addition of Subscription/Notification support for
    RPProvider RP Implementations, in response to ChinaGrid (\*1)

    -   <http://bugzilla.globus.org/globus/show_bug.cgi?id=5060>

-   Completed generalization of ESG IPs for triggers (\*1)

    -   <http://bugzilla.globus.org/globus/show_bug.cgi?id=5061>

-   Completed development of use of local transport whenever possible
    (eg communication within container), which should improve
    performance significantly (\*1)

    -   <http://bugzilla.globus.org/globus/show_bug.cgi?id=5062>

-   Completed Performance benchmarks for Clusters campaign (\*1, 2)

    -   <http://bugzilla.globus.org/globus/show_bug.cgi?id=3972>

```{=html}
<!-- -->
```
-   Fixed Bugs:

    -   Bug 5006: File configuration error does not print useful error
        message. Fixed Apache Axis to provide information on badly
        configured file to improve usability.

    -   Bug 5048: Fixed deadlock issue with JNDI objects and security
        descriptors at container startup. This feature is used by GRAM
        service for job recovery at startup.

    -   Bug 5071: Fixed build system to work with JDK 1.6.

    -   Fix for globbed ftp transfers in gass copy library when
        connection\
        fails after being initiated by a feature check (bug 3141)

    -   Fix for corrupted response messages in ftp_control library when
        it\
        gets two in one read (bug 4735)

    -   Fix for leaking connections in client library when trying to
        cache\
        duplicate source/dest pairs (bug 4344)

    -   Fix for race condition in client library on quit/cleanup with
        threaded\
        builds (bug 4945)

    -   Fixed GridFTP server process hangs when a zero length CRL is
        present\
        (bug 4825)

    -   Fixed GridFTP server listings to show status of pipes (bug 4786)

    -   Fixed a bug in globus-url-copy where globbed transfers failed if
        one\
        of the files contained a newline character.

    -   Fixed a build failure on systems that require stdint.h to
        define\
        int64_t (bug 4586)

    -   Bug 5100: Authz failure when using \"pooled account patch\" from
        gridsite.org (WORKAROUND)  \--
        <http://bugzilla.globus.org/bugzilla/show_bug.cgi?id=5100>

**Incubators**

-   Additional work defining Incubator escalation process

```{=html}
<!-- -->
```
-   Planning for Incubator review in March

**RFT**

-   Working with QCD group (from UK) in requirement analysis for RFT to
    be\
    used in their grid

**2b Total number of bugs fixed: 46**

**3. Specialized Community Support (OSG, TeraGrid, not mentioned above
including meetings (e.g. requirement interviews, planning meetings).**

-   Initiated weekly meetings to ensure support for OSG ITB 0.6.x WS
    GRAM testing

-   Assisted with existing production deployments of pre-ws GRAM, as \
    well as working with the WS GRAM ITB team to make sure WS GRAM
    meets \
    the requirements of OSG sites.

-   Ongoing participation in OSG Consortium and OSG Council meetings

-   Email discussions with Mike Wilde, OSG outreach coordinator, and Ben
    Clifford about Globus involvement in OSG campus cyber infrastructure
    work. Discussions to be ongoing. (Also funded by OSG)

-   Discussions with CEDPS team about overlap between their outreach
    work and CDIGS outreach work, including upcoming visit to ORNL (Also
    funded by CEDPS)

```{=html}
<!-- -->
```
-   MDS4 Team in ongoing discussions with TeraGrid about deployment in
    support of portals, seems to be going well, some additional
    development work defined for more esoteric resources (\*1, \*3)

-   MDS4 Team in additional discussion with TeraGrid about ways to
    gather basic resource information, similar to patch we gave them
    over a year ago which was never deployed, as users are now
    requesting this data. Forward plan still being defined, although
    there is a worry that we won't have the manpower to help as much as
    we'd like to do this in a timely fashion (\*1, \*3)

-   MDS4 Team has loosely coordinated interactions with ESG on a regular
    basis to define any upcoming trigger service work. Current campaigns
    reflect these conversations. (\*1, \*4)

-   MDS4 Team participated in OSG all hands meeting, March 5-7, SDSC,
    including meetings with OSG troubleshooting team and ongoing
    discussions with both OSG and ESG users.

-   posted information on how anyone can submit documentation (on
    dev.globus.org under \'How to contribute\')

-   \"FNAL-Globus Security Discussion\", Frank Siebenlist, Rachana
    Ananthakrishnan, Fermi National Accelerator Lab, Batavia, IL, Feb
    14, 2007.\
    OSG wants to replace its custom code and interfaces with
    standardized ones that are supported by GT. This includes XACML2
    authz query interface support and possible use of CAS. Plans have
    been made to address the missing pieces.

-   Met with the China National Grid to outline GT security framework.

**4. Incubator Projects**

-   **Gavia Meta Scheduler** -- contact Andre Charbonneau, National
    Research Council of Canada (NRC) A Condor-based metascheduler using
    the Globus Toolkit 4 as the Grid middleware.

-   **Gavia Job Submission Client** -- contact Andre Charbonneau,
    National Research Council of Canada (NRC) A generic graphical user
    interface for job submission, monitoring and management that is
    tailored to work with a Globus 4 Grid running the Gavia
    Metascheduler.

-   **OGRO** -- contact Jesus Luna, Universitat Politècnica de Catalunya
    The Open GRid OCSP (Online Certificate Status Protocol) client uses
    CertiVeR for proxy certificate's OCSP path validation and to request
    authorization information in OCSP extensions from such service. The
    OGRO project leverages the OCSP-GT-integration work by the
    Universitat Politècnica de Catalunya and Certiver
    (<http://globus-grid.certiver.com/info/ogro/>). It addresses an
    important gap in GT's feature set, and our hope is to integrate this
    software for the GT4.2 release. This project adds support for the
    Online Certificate Status Protocol (RFC 2560) to the Globus Toolkit.
    It was designed to fulfill the special OCSP's validation
    requirements imposed by Grid environments to relying parties. OGRO
    is 100% Java and can be easily configurable through the Grid
    Validation Policy (GVP), a set of XML rules that mandates its
    behavior.

-   **SJTU GridFTP GUI Client (SGGC)** - contact Linpeng Huang, Shanghai
    Jiao Tong University The SJTU GridFTP GUI Client (SGGC) is an
    interactive GUI client for GridFTP.

**5: Outreach through Scholarly works, standardization efforts,
conferences & workshops:**

**\--Publications and white papers**

-   Ionut Constandache, Daniel Olmedilla, Frank Siebenlist,
    \"Policy-driven Negotiation for Authorization in the Grid\",
    submitted to Policy2007, 26 Feb 2007 (accepted).

-   Editors: T. Mori, F. Siebenlist, \"OGSA(tm) Basic Security Profile
    1.0 - Secure Channel\", OGSA-WG, (last call at OGF)

-   Editors: T. Mori, F. Siebenlist, \"OGSA(tm) Basic Security Profile
    1.0 - Core\", OGSA-WG (last call at OGF)

-   Stephen Langella, Scott Oster, Shannon Hastings, Frank Siebenlist,
    Tahsin Kurc, Joel Saltz, \"Enabling the Provisioning and Management
    of a Federated Grid Trust Fabric\", submitted to 6th Annual PKI R&D
    Workshop 2007, Feb 1, 2007 (accepted).

-   Frank Siebenlist, Michael Helm, Rachana Ananthakrishnan, Ian Foster,
    \"Trust-Root Provisioning and Validation Facilities\", Whitepaper
    for DOE Cybersecurity R&D Challenges for Open Science: Developing a
    Roadmap and Vision, Jan 24-26, 2007.

**\--Papers and abstracts submitted to meetings and conferences**

-   GT4 GRAM: A Functionality and Performance Study, submitted to TG
    '07.\
    <http://www.globus.org/alliance/publications/papers.php#TG07-GRAM>

-   TeraGrid\'s GRAM Auditing & Accounting, & Its Integration with the
    LEAD Science Gateway, submitted to TG '07.\
    <http://www.globus.org/alliance/publications/papers.php#TG-GRAM-Auditing>

-   Tutorial proposal on deploying GridFTP and how to configure it to
    get the optimal performance, got accepted at the LCI International
    Conference on High-Performance Clustered Computing.

-   Submitted a paper on GridFTP pipelining, a solution to improve the
    performance of lots of small files transfer, to the TeraGrid
    conference.

-   Confirmation of Globus session at OGF20

-   Ongoing discussion of a day-long Globus software event at OGF21

-   "Monitoring the TeraGrid with MDS4", Jennifer Schopf, Eric Blau,
    Mike D'Arcy, Maytal Dahan, Neill Miller, Laura Pearlman, Eric
    Roberts, submitted to Teragrid 2007.

-   Von Welch , Ian Foster, Tom Scavo, Frank Siebenlist, Charlie
    Catlett, Jill Gemmill, Dane Skow, \"Scaling TeraGrid Access: A
    Testbed for Identity Management and Attribute-based Authorization\",
    submitted to TeraGrid\'07, Feb 8, 2007.

-   Yuri Demchenko, Leon Gommans, Cees de Laat, Frank Siebenlist, David
    Groep, Oscar Koeroo, \"Security and Dynamics in Customer Controlled
    Virtual Workspace Organisation\", submitted to HPDC 2007, Jan 22,
    2007.

**\--Standardization efforts**

-   Reviewed OGF Compute Area specification: JSDL, BES, HPCP, SPMD

-   Attended the weekly Telecons for OGSA-DMI working group of OGF.
    This\
    work has gained interest in OSG and EGEE and our involvement will
    help\
    us steer its direction\
    - Worked on draft specification of the above.

**\--Presentations and tutorials given**

-   \"Security Provisioning and Validation, Policy Enforcement
    Complexity and Data Integrity Verification\", Frank Siebenlist, 11th
    Middleware Security Group Meeting, San Diego, CA, March 1-2, 2007

-   \"The Earth System Grid - Security to enable Access\", Frank
    Siebenlist, NSF Cybersecurity Summit 2007, Arlington, VA, Feb 22-23,
    2007.

-   \"FNAL-Globus Security Discussion\", Frank Siebenlist, Rachana
    Ananthakrishnan, Fermi National Accelerator Lab, Batavia, IL, Feb
    14, 2007.

-   \"Identifier Services Framework\" as part of the \"caGrid 1.0
    Service Architecture\" presentation, Frank Siebenlist and Doug
    Mason, caBIG 2007 Annual Meeting, Feb 5-7, 2007, Washington, DC.

-   Security tutorial at NeSC, Edinburgh on December 13th and 14^th^\
    (<http://www.globus.org/toolkit/tutorials/javaWSSecurity/>) at EPCC

-   11th Middleware Security Group Meeting, San Diego, CA, March 1-2,
    2007

-   NSF Cybersecurity Summit 2007, Arlington, VA, Feb 22-23, 2007.

-   caBIG 2007 Annual Meeting, Washington, DC, Feb 5-7, 2007.

-   DOE CyberSecurity Workshop, Washington, DC, Jan 24-26, 2007.

-   OSG-FNAL-Globus Security Discussion, Fermi National Accelerator Lab,
    Batavia, IL, Feb 14, 2007

-   caOps meeting, Austin, TX, Nov 29-30, 2006.

-   Presented ongoing work with trigger service and archiver to CEDPS
    troubleshooting group\
    <http://www-unix.mcs.anl.gov/~schopf/Talks/mds4.cedps.mar2007.ppt>

-   Presentation on current MDS4 related projects to internal ANL/UC DSL
    management

**\--Press Releases**

-   Press release March 7 which also appears in several newsletters and
    online publications
    <http://dev.globus.org/wiki/Incubator/Incubator_AnncMar7>

**6: Additional Outreach activities or meetings.**

-   Worked on plan to gather WS GRAM Success Stories from users

-   OMII-Europe outreach \-- Working with European partners in pursuit
    of interoperability across Grid infrastructures.

-   Worked with EPCC software architect to design use of GT
    authorization framework in OGSA DAI, Edinburgh, UK, Dec 15, 2006

-   Proposed six GT projects for Google Summer of Code Project.

-   Set up framework for tracking Outreach in bugzilla
    <http://snipurl.com/1cyup>. This allows anyone to submit a request
    for Globus presentations, talks, or tutorials, and lets us track
    ongoing outreach work.

**7. Documentation & Web site related work**

-   Released documentation and release notes for 4.0.4

-   Preliminary howto index for C WS core (slowly adding more and more)

-   Began organizing for a general streamlining of docs (in
    4.2-drafts/), more howtos, and new user-friendly docs including a
    top-level commandline guide

-   Began working on new gridftp mini-site - with dedicated logos
    (similar to how apache products logo - the GT logo is still there
    and we\'re just adding \"GridFTP\" to it), this could turn into a
    good thing for all components (more of a portal).

-   Continued with infrastructure buildout for GT manuals project and
    new Incubator projects.

    -   Added review examples\
        <http://dev.globus.org/wiki/Incubator/Incubator_Process/Review-2>

    -   <http://dev.globus.org/wiki/Incubator/Incubator_Process/Escalation_review>

    -   Interface table for message security

    -   General wiki updates for new incubator projects

-   General website updates (news, papers, etc)

-   General docbook and writing assistance

-   Continue updating admin guide in response to community feedback.  

-   Worked on a new user\'s guide, and administration guide for the
    Workspace client

8\. What two or three key items do you have planned for next month (may
include ongoing activities)?

-   Preparations for GT 4.0.5 release (all technologies including
    updated docs)

-   Produce a WS GRAM development plan for support of OGF compute
    specifications

-   Continued evaluation and assessment of some competitor products to
    provide recommendations on future plans for the Java WS Core
    component

-   Plan for Outreach work over the next year with several key
    communities

-   Incubator Review planned for third week of March

-   Integrate the XACML2-SAML2 authz call-out interface in the
    GT-runtime and into the CAS service. This will address OSG's
    requirements to replace their custom authz query interface with a
    standardized and supported one.

-   Finish the re-factoring of the AuthZ framework such that it also can
    be used on the application-level (outside the WS-stack).\
    This will greatly enhance the applicability of the authz-framework's
    functionality.

-   Enhance CAS such that it can easily be deployed as a
    black/white-list service.\
    This will also address OSG and possibly TG's gateway's requirements.

-   Work on updating RLS metrics and analyzing collected information

-   Begin adding metrics to Data Replication Service

-   Streamlining 4.2-drafts

-   Develop howtos/indices

-   Finish GridFTP mini-site and start making similar mini-sites for
    other components (thinking gram4 next)

\* In addition to CDIGS, some accomplishments funded in part by:

1\. NMI: Performance Inside: Performance Monitoring and Diagnosis for
NMI Software and Applications

2\. ITR: Image-based Biophysical Modeling: Scalable Registration and
Inversion Algorithms and Distributed Computing

3\. TeraGrid

4\. DOE: SCIDAC