Sunsetting my Work at Notre Dame

Sunsetting my Work at Notre Dame

I worked in the Office of Information Technologies at the University of Notre Dame from 1999 to December 2004. As an Information Engineer one of the primary systems I was responsible for designing, implementing, and supporting was the Enterprise Directory Service we named "EDS".  The EDS was essentially an LDAP directory with a schema intentionally designed to support multiple applications for authentication, authorization, and attribute release, as well as providing enhanced user lookup for email clients and directory search web pages. Perl scripts were used to transform and import data from a variety of sources into the EDS and other scripts were used to provide group management and data exports for downstream services.

I've learned that while much of my EDS work has remained available and functioning since implementation almost 20 years ago, the system in its entirety will be replaced by a vendor product this fall. This blog post describes some of that work. For the moment it is still accessible at https://eds.nd.edu.

The EDS home page from 2004 to 2019

In addition to leveraging my own background in data modeling, my LDAP schema design was influenced by the "LDAP Recipe" of Michael Gettes, white papers on identity written by Bob Morgan, the early eduPerson object class work being done by the MACE-Dir working group (which I joined in 2000), "Understanding and Deploying LDAP Directory Services" by T. Howes, M. Smith, and G. Good, and assorted other texts. I was fortunate to have a good working partner in Jeremy McCarty and together we were able to stand up a robust, capable, and reliable service in early 2001 after several months of study, analysis, experimentation, and testing.

Initially the schema attributes that were populated were designed to provide a replacement for Notre Dame email address lookup for the campus and peer institutions. Indexing decisions and ACLs (Attribute Control Lists) were designed carefully to provide just the right information to just the right people or applications, in just the right way. It is fair to say that my design pushed the ACL flexibility of the iPlanet Directory Server to the edge. It would not have been possible with any other product. Later in 2002 attributes were added to support fine-grained authorization using groups, and then in 2003 enhancements were made to support integrations with the campus Microsoft Active Directory Service and SendMail. The final update to the schema documentation was published August 6, 2004 though subsequent schema modifications were made after I left in December of 2004.

From the very beginning though, during early design in 2000, it was necessary to be able to query the EDS and show the data and validate that not only was the data populated correctly for entries, but that the ACL's and indices were properly configured to search and release the correct entries and attributes. To that end I spent time creating a web site at eds.nd.edu to house first a search application and later online documentation and self-service privacy controls and email preferences.

At the time OIT had a team that used Dreamweaver to generate HTML sites and they wanted all OIT departments to utilize their templates so that there would be a common look and feel across our services. I embraced that idea and developed templates that included embedded functions that the Perl cgi program nd_ldap_search.pl would execute when rendering the pages. In this way I could readily leverage the common templates while still providing the customizable interface I wanted without having to constantly rewrite the Perl cgi code that generated the pages. The Perl code could be focused on the functions and the Dreamweaver template could handle the static web page content. All of this required a lot of HTML, Javascript, and Perl work and making sure that each layer was doing its part.

The EDS Search page

The best example of this collaborative architecture is the EDS advanced search page. Most searches performed at the time were based on simple common attributes such as name, affiliation, department, and the unique University NetID which was an AFS id. Notre Dame has always allowed people to have multiple email addresses, so searching by email address was also frequently needed.  The simple search page allowed these searches to be done on the web. The attributes specified for searching in the HTML form would be passed to the nd_ldap_search.pl Perl script which would construct the necessary LDAP search filter, execute the search against the EDS directory, and then construct a response page which could display one or many entries. Simple searches required only simple LDAP filters to be constructed, but the LDAP protocol allows for very complicated filters and so an Advanced Search page was also provided that could take full advantage of LDAP filter potential.

With the Advanced Search page extremely complicated search filters could be created to select entries and a large number of attributes could be returned.

When only a single entry was returned the results page was constructed to resemble a page that would be returned by a standard iPlanet search when using their UI tools.  This was done to minimize user impact if we ever decided to utilize the vendor tools rather than our own (we never did).

Clicking on the "Display Complete List of Attributes" link at the bottom would cause the search to be rerun but return the results as a list.  There were a number of other formats to return the attributes as well, including as raw LDIF. Of course any entries or attributes that were restricted in any way would not be returned. Entries and attributes could be set to public, private, or restricted to the ND network. This was very important in order to ensure compliance with FERPA.

If multiple entries were returned by the search then they would be displayed in a list with minimal attributes displayed in columns and then clicking on an entry in the list would in turn display the single entry in detail as shown above.  This same ability to display entries collectively was extended to support searches against multiple directories so that it was possible to initiate a single search for a person across as many public LDAP directories as desired. This would have been a convenient way to find colleagues at other institutions.

Full documentation was included on the site to aid in accessing the EDS and utilizing its capabilities, including how to create department specific templates and search pages, though this capability was, as far as I know, never used.

There was also a series of pages available only to authenticated users that allowed authenticated searches and email preferences to be specified for spam routing, white listing, black listing an other capabilities supported by Sendmail using sieve. There were also group management pages created so that authorization could be controlled by groups.  Unfortunately all of these pages are no longer accessible to me and I do not know which of those functions continue to be used. A search through my presentations from those years includes information about those capabilities at that time.

When I first interviewed with Notre Dame in 1999 I told the hiring manager, Gary Dobbins, that I wanted to build something that would last. While it is sad to learn that these pages and scripts will finally be retired later this year, I am proud that my work lasted in production nearly 20 years and I doubt that its replacement, no matter how expensive, will fare as well.