Joining the AEON Infrastructure
The AEON Network is delighted to welcome new partners, and is happy to work with telescope operators outside of the US and Chile in the context of the LSST In-Kind Proposals process.
A key goal of the network is to allow partner facilities to integrate seamlessly when they choose to operate in AEON mode, and to operate autonomously when they prefer.
For context, a short overview of the technical requirements for joining AEON can be found here.
As each facility has unique circumstances and capabilities, determining the best way to ensure compatibility with the network varies in each case. We recognize that many facilities will already offer advanced capabilities, but it is our goal to make AEON accessible to a wide range of facilities, starting from a similarly wide range of current capabilities. The purpose of this document is therefore to lay out the technical options as clearly as possible, with questions designed to guide each facility as they consider which option is best for them, and gauge the level of work involved.
These questions are broken out into the following sections:
- Scheduling and interfaces
- Data Reduction and Distribution
- Documentation and User Support
- Time Allocation Process
- Development and Direction AEON
- Example AEON Case Studies
- Example 1: Manually-operated 2m optical telescope with a spectrograph, taking advantage of the LCO scheduler 6
- Example 2: Manually-operated 8m telescope with multiple instruments provides an AEON-compatible interface
Teams developing LSST in-kind proposals that are interested in AEON partnership are encouraged to contact the AEON representatives (as listed here). AEON is able to offer endorsements of proposal elements but only if the details are discussed in advance of the proposal deadline.
Scheduling and interfaces
AEON observations must be conducted in queue-scheduled mode, as this offers users of a facility with the greatest flexibility to conduct observations at the most optimum time for their science.
AEON defines a queue-schedule to be a dynamic process for generating the sequence of observations to be carried out at a given telescope that allows observations from different programs to be interleaved. Although different implementations are possible, a key characteristic of a dynamic queue-scheduling system is the ability to update the observing sequence to accommodate new observation requests within a finite timescale. A scheduler enables the widest range of observations if this timescale is short, ideally less than 24hrs. Once an observation request sequence is received or generated by a telescope, the requests can be carried out robotically or by observatory staff, according to the facility’s normal operations.
This raises some operational considerations for facilities joining AEON. Questions to answer are:
Q) Which instruments will the observatory offer in AEON mode?
Q) Is the facility robotically controlled or human operated?
Q) Can it accept target of opportunity requests requiring a rapid response? If so, what is the mechanism for this?
Q) What information does the facility need in order to carry out a requested observation? This needs to include all aspects of instrument and telescope configuration, scheduling constraints, target information (possibly including a finder chart), guiding information, calibration requirements.
Q) How will AEON-mode time fit in with the other time allocations on this facility? This is ideally achieved by the facility operating under a dynamic queue system at frequent and regular intervals throughout each observing semester, if not all the time. It might also be achieved by allowing flexible Target-of-Opportunity observations to occur at any time.
Q) What calibration data are required in order to reduce data from the instruments offered? How will these calibration data be obtained during queue-mode operation? Through a standard program operated by the observatory at suitable intervals, or is each user responsible for obtaining the calibration data necessary for their observations?
If robotically controlled:
Q) What kind of scheduling system controls the sequence of observations? [E.g. dispatch scheduler, pre-determined survey sequence…]
Q) How would this system respond to queue-mode requests received from AEON? What limitations are there?
Q) Does this system already offer an API for externally submitted requests? If so, an interface layer to interpret AEON-format requests into the facility’s existing request format may be the simplest implementation.
If human operated:
Q) How does the observatory organize staff to be available to conduct queue-mode observations during AEON-mode periods?
AEON currently offers two distinct pathways for a facility to receive observation requests:
- Integration with the Las Cumbres Observatory scheduler, activated at the discretion of the facility owner [example: SOAR]
- Independent interface capable of receiving observation requests programmatically submitted using the AEON request language. [example: Gemini]
Integration option 1: Integration with the Las Cumbres Observatory scheduler
This option enables an optical/infrared facility to take advantage of the existing software infrastructure provided by Las Cumbres Observatory. This paper describes the concept in more detail. Facilities interested in this option should contact LCO directly to determine the cost for LCO implementation and operations. In the case of LSST In-Kind Contributions, the applying facility would cover these costs. This includes an online portal and API for observation submission, and a responsive and highly flexible queue scheduling system which provides a sequence of observations for each facility, including a rapid response capability. It may also include integration with online data archives, though this is not the default. For example, SOAR data taken in AEON-mode is disseminated to both the NOIRLab and LCO data archives, and links to the products of each observation are provided through the observation request portal. A similar arrangement can be made for other facilities, where the costs of hosting their data are borne by the facility concerned. The system is fully supported by the TOM Toolkit, so the new facility would not need to build any user interfaces.
To take advantage of these systems, the new facility would need to build APIs to share the necessary information with the Las Cumbres system. These APIs provide the functions:
- Indicate whether or not the facility is available for AEON observations
- Receive sequences of observation requests from the Las Cumbres scheduler
- Indicate the status of execution of those requests
- Allow configuration of the resource available for scheduling, e.g. instrument modes
- Receive information about proposals and user teams
- Indicate current cbserving conditions for scheduling, e.g. seeing (optional)
- Provide a queue for shipping data products (raw & reduced) to science archive (optional)
A more detailed, though now dated, description of the LCO-SOAR interfaces can be found here. Please note that this was intended for developers and is not currently maintained.
LCO has recently made its observatory control software available as an open-source standalone package, including many of the features necessary to integrate with AEON. Observatories are encouraged to explore this option, and to contact LCO for further information.
Q) How are observations carried out at the facility?
Once observation requests are received the facility can execute them in queue mode following their normal operational procedures. The facility may need to develop software to enable observatory staff to display information on the observations requested, instrument status, or (if these functions are carried out programmatically) to develop an interface to communicate the observation requests received to the observatory control system.
Integration option 2: Independent interface
Some facilities already have programmatic interfaces, or prefer to develop their own. In this case, the facility will need to provide an API capable of receiving and interpreting observation requests in the AEON standard format, and to indicate the status of the observatory (online, closed for weather etc). Ideally, it would also provide interfaces to indicate the status of execution of each request.
If a facility choses this option, they should provide an application to enable users to submit observations to their facility using the TOM Toolkit. An example of this kind of application has been developed for the Liverpool Telescope. An interface for the Gemini Telescopes is provided within the base TOM Toolkit. The TOM Toolkit developers can advise on developing this interface. In the context of LSST In-Kind Contribution, all development costs would be covered by the applying facility.
Data Reduction and Distribution
Once observations are requested at a facility, the resulting data products should be made available to the user who requested them as quickly as possible, ideally within 24 hrs. At minimum, the data made available must include the raw data products and all calibration data necessary to fully reduce the science data. Delivery of reduced data products is not a requirement for AEON, but highly recommended. It is also recommended that users be able to access their data products programmatically, through an interface which respects proprietary data controls.
Q) What types of data products will be provided?
Q) What is the format of these data products?
Q) How and where are these data products archived?
Q) How will users access their data? Does this offer a programmatic interface?
Q) How quickly will users be able to access their data?
Documentation and User Support
Even when a facility operates entirely robotically, good user support is essential, starting with thorough and accessible documentation describing the facility, its instrumentation, observing modes, calibrations and user interfaces. The documentation should be publicly available online, and should also include all information on exposure time calculations, instrument overheads etc necessary for astronomers to accurately plan observing programs and prepare proposals. AEON facilities are responsible for supporting users of their facility, and should respond to all questions concerning its instrumentation, operational and interfaces. If a facility chooses to integrate with AEON through the Las Cumbres network, Las Cumbres staff will handle inquiries regarding its online portal, scheduler and API. Documentation is particularly important if a facility decides to develop their own AEON interface. An example of API documentation can be found here.
Q) What documentation is available for the facility, its instrumentation, calibration, user interfaces (online and programmatic) and data archive?
Q) What documentation will observatory staff need to operate in queue mode, or to monitor robotic operations?
Q) User questions inevitably arise even with comprehensive documentation. Where can a user find information on who to contact? Who will respond and how quickly?
Time Allocation Process
There are advantages to AEON-mode time on participating facilities being administered through a joint Time Allocation Committee (TAC) process. AEON is particularly advantageous for users who need to use multiple facilities to achieve their scientific goals and a joint TAC process allows them to describe their strategy most clearly, while minimizing the overheads associated with proposal writing and evaluation.
In particular, telescope time contributed through the LSST In-Kind Contributions process will be administered through the NOIRLab TAC process.
Eligible users will compose and submit proposals through the NOIRLab system, and evaluation will be conducted through their existing TAC program.
A participating facility may be called on to provide technical advice regarding the feasibility of proposals during the TAC process.
Q) How will technical review of proposals be performed?
Q) Who will provide technical advice?
Information on successful proposals will be returned by NOIRLab to each facility. This information will then need to input into the system that provides the user access to their telescope time. For facilities that are integrated through the Las Cumbres Observatory interface, NOIRLab will share details of successful proposals with Las Cumbres, whose staff will perform the necessary preparation for users to access their time. NOIRLab will also share this information with facilities that opt to develop their own interface, but those facilities will be responsible for configuring their interface to accept the proposals, add user accounts etc.
Q) What procedure do you have for notifying NOIRLab regarding the time to be made available for proposals? This should be done in good time before the call for proposals is issued. Information on the NOIRLab semesters can be found here.
Q) What information is required by the facilities proposal system in order to configure a new active proposal and users?
Q) Who at the observatory is responsible for overseeing this process?
Q) Successful proposal PIs will be notified by NOIRLab, but observatories are responsible for notifying users of additional information needed to request and execute observations. How will users be notified?
Development and Direction AEON
AEON is managed by a team of representatives of the partner facilities. As of Aug 2020 these include representatives from the founding institutions: NOIRLab, Gemini, SOAR and Las Cumbres Observatories. New partners will be invited to nominate a representative to join this team.
Example AEON Case Studies
The following case studies are provided as step-by-step examples to illustrate how two different facilities might interface with AEON. It is worth noting that the infrastructure is highly flexible, so alternatives can be considered in collaboration with the AEON partners.
Example 1: Manually-operated 2m optical telescope with a spectrograph, taking advantage of the LCO scheduler
In this example, the telescope is normally operated in a block schedule, where PIs are normally awarded blocks of a few nights back to back. The telescope is run by a small staff who maintain the facility, and provide support to visiting astronomers who typically conduct their observations in person.
- Software development effort to build APIs to communicate with LCO
- An estimated 0.5FTE will be needed to develop the interfaces outlined above in collaboration with LCO. Templates of existing code can be provided as a guide. Minimal development work required at LCO as this instrument class is already supported
- Documentation and support
- Facility staff will need to fully document all aspects of the facility, instrument and operations.
- Staff will need to maintain the documentation and answer user inquiries
- Allocate a pre-agreed number of nights to AEON-mode queue-scheduled operations
- These should be distributed regularly throughout each semester, but the facility can be scheduled normally at all other times.
- Facility staff operate telescope in queue mode on AEON nights
- Telescope operators will indicate when the facility is in AEON mode through the software interface with LCO, and the LCO scheduler will respond with a sequence of appropriate observations.
- Telescope operators obtain bias, dark and flat field observations, as well as arc lamp exposures appropriate to each observation.
- Telescope operators conduct those observations throughout the night, and use the software interface to indicate when each one is complete. LCO scheduler will respond with an updated sequence of observations.
- Telescope operates upload the data products resulting from the observations as quickly as possible through the facility’s own data archive. The PI of the proposal is enabled to access the data.
- At the end of the AEON night(s), operators return the telescope to normal operations
Example 2: Manually-operated 8m telescope with multiple instruments provides an AEON-compatible interface
In this example, the telescope is normally operated in queue-scheduled mode, where facility staff operate the telescope, and astronomers operate the instrument in real-time through a remote interface. Facility staff maintain the telescope, instrument and documentation.
The facility operators have their own operating system and scheduler and have decided to make an AEON-compatible interface.
- Facility software engineers develop programmable interfaces to their system that allow the submission of observation requests for their instruments.
- Facility software engineers develop a plugin for the TOM Toolkit that enables TOM users to submit observation requests to their facility’s interface.
- The facility’s own scheduling process ensures that these requests can be executed at the telescope at appropriate times.
- Observatory staff conduct the observations during regular night operations and ensure the resulting data products are made available to the user through their data archive.
- Facility staff maintain up to date documentation, and answer user inquiries.