Tree Closing Window Planning¶
The purpose of the Planning Procedure below is to state the de facto agreement on the process for Tree Closing Windows (TCW). A few enhancements will be proposed along the way in italics.
The goal is to have all the workload, durations, and personnel assignments done at least one week in advance of the TCW.
All requests for work during a TCW must be submitted to CAB no later than 2 meetings prior to the TCW. (This is typically 10 days.)
Late requests that extend the TCW will be denied unless they are emergency.
Late requests that fit within the planned window will be denied unless there is good reason for the lateness. Even so, they may be denied if personnel changes would be needed.
Late requests that do not require a tree closure are up to IT to decide.
All requests must include (handled as part of the CAB Change Request process):
Steps and duration for “all goes well”.
Steps and duration for rollback.
Assistance needed (specifically). E.g. “zeus expert” or “buildapi expert”
There is no official calendar of potential TCW dates (in part because there is no official calendar of release dates). A TCW date is calculated as follows:
The master schedule is driven off the official release schedule. Usually, TCW is the 3rd Saturday after Firefox release, but may be the 4th with pre-arrangement with Release Management.
No later than 2 Fridays preceding the TCW (8 days), the composite time line will be established and documented in a spreadsheet.
The tracking bug summary should be updated with the planned start and completion times of the TCW.
An early view of what has been requested for the next TCW can be seen via the TCW Requests link into Service Now.
As soon as the list of activities is determined, RelEng can make a determination whether the TCW can use a “soft close” or a “hard close”.
No later than end-of-day Wednesday preceding the downtime, RelEng constructs an email announcement, based on the text posted on internal status by MOC. That email is cross posted to the appropriate engineering lists. (Currently, that is dev-planning, dev-tree-management, dev-platform.)
RelEng Contact Posts to Newsgroups
The MOC will publish a note on the internal status page describing the changes. However, they do not have posting privileges to the newsgroups. Therefore the RelEng contact person posts (forwards) on the MOC’s behalf, adding any addition information (such as “soft close” status). Even if the MOC email is sent to everyone, we still post to newsgroups to reach community members of the development community.
Execution of TCW¶
The MOC administers the TCW. In general, the following generally occur:
Spreadsheet with name of person on the hook for each bug and support group created. Links and any special contact info should be in the pad.
Planned timeline in the spreadsheet.
Communication channels defined. Usually irc (#moc), backup irc (freenode #mozilla-it), Vidyo (MOC) if needed.
Designated person to run the TCW. Usually the on-call MOC staffer.
Spreadsheet is kept current with status, including times.
All people involved with a tree closing part of the window remain on call until the trees are reopened.
See detailed checklist for actions to be taken.