port rev-upgrade

Clemens Lang

Short description: If a library upgrade changes the library API dynamic linkage of software depending on it will usually break. In order to fix these problems, the broken binaries need to be identified, matched to ports and these ports need to be recompiled. This corresponds to reversing the dependency direction, which is why this action is called rev(erse dependency)-upgrade. This step should be done automatically right after the library upgrade instead of manually by the user as soon as the problem is noticed.

Personal

Name: Clemens Lang
E-Mail: neverpanic@gmail.com ^{(PGP key)}
IRC/IM: neverpanic on FreeNode and IRCnet
neverpanic@gmail.com on Jabber/GTalk
University/Enrollment: University of Erlangen-Nuremberg
Location/Timezone: Erlangen, Germany; GMT+1
Short Bio: I'm currently enrolled in my 6th semester of computer science at the University of Erlangen-Nuremberg. During the last few years of school I became interested in programming for the web, learning both PHP and JavaScript. After graduating I completed two internships in this field and later registered for computer science at my current university. My focus has since shifted from the web to more general languages used in computer science, such as Java, C, C++ and some Bash scripting. I also extended my knowledge of UNIX and Linux systems and am currently administrating a number of boxes running Debian, Ubuntu and Gentoo Linux.
Programming knowledge: I speak Java, C, C++, PHP, JavaScript and Bash Scripting fluently and have some experience in ActionScript (Flash), Python and Objective-C. I know the Waterfall- and V-Model software engineering methods and have experience in Scrum. Design patterns ease my work where applicable. Source control tools (mainly subversion and git) are as common in my usual workflow as Make and CMake.

Project

Abstract

Upgrading libraries can be a tedious task – if the library API changes with the upgrade it's version number and with it it's filename change. This breaks dynamic linkage for binaries depending on the upgraded library. It is this project's task to

detect such errors, and
fix them by rebuilding the package the broken binary belongs to.

Because such breakages happen in reverse direction of a dependency this is called a reverse dependency rebuild (think revdep-rebuild on Gentoo) or upgrade (hence the name rev-upgrade for the suggested task).

Broken dynamic linkage can be detected using the Apple-provided otool. A naive approach to find broken binaries could be implemented by iterating over all binaries in the MacPorts prefix, calling otool -L, and parsing the output. Files found to be broken must then be matched to their corresponding port, which can be done using MacPorts' registry SQLite database. These ports can then be rebuilt using the existing port command.

Details

I suggest a subcommand to be added to the port command named rev-upgrade. This command would scan the MacPorts prefix for linkage errors, compile a list of ports and rebuild them. This command could be run automatically after upgrading ports.

Roadmap

Community bonding period (April 27^th - May 24^th, 28 days)

set up development environment
read up on Tcl
study MacPorts sources (coding style, useful functions, possible entry points)
add empty rev-upgrade subcommand to port

1^st coding period (May 25^th - July 12^th, 49 days)

14 days: naive scanning using otool -L to determine broken files
10 days: match broken files to ports
14 days: rebuild these ports
11 days: test and handle corner-cases (system_x11, multiple architectures)

Result: Working but probably slow implementation of the rev-upgrade command

2^nd coding period (July 17^th - August 16^th, 31 days)

14 days: optimize performance by reimplementing the necessary parts of otool in
- Tcl, or
- C and calling it from Tcl, or
- a C program to do the scan and print a list of broken files
and caching results
5 days: run rev-upgrade automatically after each upgrade
7 days: write test cases and fix bugs
5 days: write documentation

Result: Fast, tested and documented implementation of the rev-upgrade command

The remaining time between suggested 'pencils down' and firm 'pencils down' serves as buffer for unexpected changes and delays.

Additional Questions

What are your experiences with Mac OS X so far? / How long have you been using MacPorts?

I got my first Mac for university, and have been using it about 2.5 years now. I was missing some open source software I had gotten used to and while looking for an easy way to install them other than the usual ./configure && make && make install I discovered and tried Fink and MacPorts and decided to stay with MacPorts.
Besides my Mac I'm also using several Linux desktops (mainly at university) and servers.

Do you have experience with other package management systems?

I am familiar with Debian-based package management using apt/aptitude and Gentoo portage.

Have you already made contributions to MacPorts?

I have reported a few build errors to MacPorts.

Have you already made contributions to other Open Source projects?

I have published GeSHify, one of my projects under an open source license.

How much experience do you have with Tcl and C?

I have no experience with Tcl but am generally familiar with scripting languages. I have learned C at university, finished the course with the best possible grade and am tutoring C to undergraduates.