AI for space robotics auto-diagnostics and early failure prediction

The Canadian Space Agency (CSA) is seeking a solution that will decrease the cost of managing the maintenance of space robotics by making use of the large body of stored system data records to train an agent to diagnose and predict failures of the flight hardware.

Challenge sponsor:
Canadian Space Agency (CSA)

Funding mechanism:
Contract

Opening date:
September 2, 2022

Closing date:
October 28, 2022 14:00 Eastern Time

Please refer to the tender notice for this challenge on CanadaBuys.

 

Challenge
 

Problem statement

Under this challenge, a solution is sought to improve the efficiency of a maintenance program for space robotic sub-systems such as cameras, sensors, and mechanisms. Predictive maintenance (specifically self-diagnosis and failure prediction) is not a new area of research, but with advancements in machine learning (ML) and big data analytics (BDA), predictive maintenance can result in significant improvements in reliability, prediction of servicing needs based on equipment performance patterns, and reduction of equipment downtime.

CSA would like to use AI-based predictive maintenance to minimize on-orbit downtime, have longer equipment life, and reduce safety hazards, by servicing equipment based on actual wear and tear instead of scheduled service visits. The solution would make use of historical maintenance data records and downlinked telemetry from past operations as its training data (CSA can provide this data to successful bidders). By providing auto-diagnostics and early failure prediction, both preventive and preparatory actions can be taken. CSA believes that by employing new digital technologies like ML and BDA, the life cycle cost of maintaining future robotic systems can be greatly reduced.

Desired outcomes and considerations

Essential (mandatory) outcomes

The solution must:

  • Provide auto-diagnostics and failure prediction based on learning, hypothesis, and analysis using available telemetry data of a space robotic system.
  • Provide a data classifier and labeling tool which includes preparation and cleansing to ensure the data is correctly formatted.
  • Handle imbalanced data (i.e., nominal operations will be much more frequent than off-nominal).
  • Provide a diagnosis to help isolate faults in the system.
  • Predict failures of the subsystems or components of the flight system.
  • Have an adequate training method using a body of data to reliably diagnose and predict failures of the flight hardware.

Additional outcomes

The solution should:

  • Be applicable to other types of (non-robotics) equipment.
  • Include information technology security as part of its inherent design.

Background and context

The Mobile Servicing System (MSS) is an important asset for the ISS program, and its reliability is key to the ongoing life cycle of the station. For future missions like the cislunar Gateway as an example, advanced space robotics systems will operate in the harsh deep space environment and the launch cost of spare parts will be greatly increased. The reliability, safe functioning, and operating cost of these robotic systems will be key to the success of the program. The same applies to surface mobility systems such as rovers and commercial in-orbit services and active debris removal systems.

As complex systems operate in space, standard maintenance and fault diagnosis techniques might not be sufficiently cost-effective. Typical diagnostic and early failure prediction methods require extensive experimentation and modeling during the initial system development. Characterizing the on-orbit performance of the system while still in Earth gravity and ambient conditions is unfeasible, as it would require too much engineering analysis and high cost. Modern approaches based on ML and BDA could provide a solution to overcome the shortfalls of a standard maintenance management program.

At the time when MSS was developed in the late 1990s, the concept of auto-diagnosis was not highly developed and there was an absence of related technologies; hence, traditional maintenance approaches were adopted. As a result of launch schedules and other considerations, the MSS servicing was based on a 2-year cycle with 1 year mitigation period. Note also that the ISS crew is partially responsible for diagnostic and maintenance activities. The MSS operation logs report that a fairly wide range of time was spent diagnosing and correcting anomalies; examples varied from 5 minutes to several hours – and some anomalies are not yet understood. The opportunity cost of this time delays in the execution of science experiments, and potential safety hazards. This finding points to a need for a well-founded and effective self-diagnosis and failure prediction approach.

The steps in a generic use case where an agent (either a ground control operator or an onboard intelligent software agent) performs diagnosis on a space flight system would be as follows:

  1. A fault occurs in the flight system; evidence of this fault exists within the system telemetry reported to the agent.
  2. The operations context is noted; essentially this is keeping track of the intent of the current operation and any error identifiers which occur, which can help to refine the search for relevant potential faults.
  3. If necessary, the agent can choose to query the flight system for additional diagnostic information.
  4. Using the available information, a likely fault diagnosis is determined. Based on this diagnosis, the flight system can be queried again to confirm the fault, and/or obtain additional data to characterize the fault or search for a root cause.
  5. If the fault can be cleared by a telecommand, then the agent sends the appropriate command(s).
  6. The agent verifies that the fault has been correctly repaired. This may involve specifically querying the flight system for additional diagnostic information.

Another scenario of interest is autonomous detection of outside-of-nominal performance. While operating autonomously without a communication link to the ground segment:

  • The robot performs a routine operation (e.g., insertion of a payload into a receptacle).
  • An intelligent agent compares the telemetry reported by the robot throughout the operation to a model based on previous similar operations.
  • The agent determines if the performance is within normal variations, or if an anomaly or performance degradation has occurred. If the latter, the data is flagged for priority download at the next communication window so that ground personnel can perform a detailed post-analysis.

Predictive maintenance is a concept that is applied to optimize asset maintenance plans through the prediction of asset failures via data-driven techniques. The success of these approaches, such as ML and BDA, depends on having a large body of representative data. In addition, the data must be labeled correctly in order to accurately predict failure patterns or perform auto-diagnosis. Much of the data quality challenges can be addressed by deep learning algorithms that can be used to build more accurate predictive models. These deep learning models will be able to apply insights from previously labeled data to new, unlabeled data so both predictive and prescriptive analyses will become even more accurate over time.

In the future, the Industrial Internet of Things (IIoT) and Deep Learning are expected to play a substantial role in the advancement of predictive analytics and overcome data quality issues and the technology gap. Potential innovative solutions could be considered for Canadarm3 on the cislunar Gateway as well as for other commercial in-orbit and active debris removal services and surface mobility systems; Canada could lead the way in an effort to implement an intelligent maintenance program for space flight systems.

Maximum grant value and travel

Multiple contracts could result from this challenge.

Phase 1:

  • The maximum funding available for any Phase 1 contract resulting from this challenge is : $150,000 CAD excluding applicable taxes, shipping, travel and living expenses, as required.
  • The maximum duration for any Phase 1 contract resulting from this challenge is up to 6 months (excluding submission of the final report).
  • Estimated number of Phase 1 contracts: 2

Phase 2:

Note: Only eligible businesses that have successfully completed Phase 1 will be invited to submit a proposal for Phase 2.

  • The maximum funding available for any Phase 2 contract resulting from this challenge is : $1,000,000 CAD excluding applicable taxes, shipping, travel and living expenses, as required.
  • The maximum duration for any Phase 2 contract resulting from this challenge is up to 24 months (excluding submission of the final report).
  • Estimated number of Phase 2 contracts: 1

This disclosure is made in good faith and does not commit Canada to award any contract for the total approximate funding. Final decisions on the number of Phase 1 and Phase 2 awards will be made by Canada on the basis of factors such as evaluation results, departmental priorities and availability of funds. Canada reserves the right to make partial awards and to negotiate project scope changes.

Note: Selected companies are eligible to receive one contract per phase per challenge.

Travel

For Phase 1, a successful bidder may need to travel to the Canadian Space Agency (Longueuil, QC) for the final review meeting.

Kick-off meeting

Teleconference/videoconference

Progress review meeting(s)

Teleconference/videoconference

Final review meeting

Longueuil, Quebec (anticipated), or teleconference/videoconference. All other communication can take place by telephone, or videoconference.

Eligibility
 

Solution proposals can only be submitted by a small business that meets all of the following criteria:

  • for profit
  • incorporated in Canada (federally or provincially)
  • 499 or fewer full-time equivalent (FTE) employeesFootnote **
  • research and development activities that take place in Canada
  • 50% or more of its annual wages, salaries and fees are currently paid to employees and contractors who spend the majority of their time working in CanadaFootnote **
  • 50% or more of its FTE employees have Canada as their ordinary place of workFootnote **
  • 50% or more of its senior executives (Vice President and above) have Canada as their principal residenceFootnote **

Evaluation criteria

The official source of the Evaluation Criteria for this challenge is the Government Electronic Tendering System (CanadaBuys) https://buyandsell.gc.ca/procurement-data/tender-notice/PW-20-01003386)

In the event of a discrepancy between the information below and the information published on CanadaBuys, CanadaBuys will take precedence.

The Bidder must complete the Challenge Stream Electronic Submission Form with a degree of information sufficient to enable Canada's assessment of the proposal against the criteria and the Evaluation Schema. The information must demonstrate how the proposal meets the criterion.

Part 1: Mandatory Criteria

Proposals must meet all mandatory criteria identified by achieving a "Pass" in order to proceed to Part 2. Proposals that do not meet all mandatory criteria will be deemed non-responsive and given no further consideration.

Mandatory Criteria

(Bidder's proposal must address)

Question 1 a: Scope

Describe the proposed solution and demonstrate how it responds to the challenge. Include in your description the scientific and technological basis upon which the solution is proposed and clearly demonstrate how the solution meets all of the Essential Outcomes (if identified) in the Desired Outcomes section in the Challenge Notice.

Evaluation Schema (Mandatory — Pass/Fail)

Pass

The Bidder's proposed solution is clearly articulated, within the scope for the challenge and addresses all Essential Outcomes (if identified) in the Challenge Notice.

Fail

The proposed solution is articulated as out of scope for the challenge.
OR
The proposal does not clearly demonstrate how the proposed solution addresses all Essential Outcomes listed in the challenge.
OR
The proposed solution is poorly described and does not permit concrete analysis.
OR
There is little to no scientific and/or technological evidence that the proposed solution is likely to meet the challenge.

Question 2: Current Technology Readiness Level (TRL)
  1. Indicate the current TRL of the proposed solution. (Drop Down Menu of the Challenge Stream Electronic Submission Form)
  2. Describe the research and development activities that have taken place to bring the proposed solution to the stated TRL.
Evaluation Schema (Mandatory — Pass/Fail)

Pass

The Bidder has demonstrated that the proposed solution is currently between TRLs 1 and 6 (inclusive), and provided justification by explaining the research and development (R&D) that has taken place to bring the solution to the stated TRL.

Fail

The Bidder has not provided sufficient evidence to demonstrate that the current TRL is between 1 to 6 (inclusive) including:

  1. There is insufficient/no evidence provided for TRL judgment.
  2. The solution involves the development of basic or fundamental research.
  3. The solution is demonstrated at TRL 7 or higher.
  4. Insufficient/unclear/no justification explaining the R&D that took place to bring the solution to the stated TRL.
  5. The explanation simply paraphrases the description of a given TRL level.
Question 3a: Innovation

Demonstrate how the proposed solution meets one or more of the ISC definitions of innovation below:

  1. An invention*, new technology or new process that is not currently available in the marketplace.
  2. Significant modifications to the application of existing technologies/components/processes that are applied in a setting or condition for which current applications are not possible or feasible.
  3. An improvement in functionality, cost or performance over an existing technology/process that is considered state-of-the-art or the current industry best practice.

* An "invention" is defined for the purposes of ISC as: "A manufacturing design or any other new and useful improvement that is new or novel, that is, not commonly known or not an obvious derivative of an existing way of doing things."

Evaluation Schema (Mandatory — Pass/Fail)

Pass

The Bidder has demonstrated that the proposed solution meets one or more of the ISC definitions of innovation.

Fail

The Bidder has not provided sufficient evidence to demonstrate that the current TRL is between 1 to 6 (inclusive) including:

  • Bidder has not provided sufficient evidence to demonstrate that the proposed solution meets any of the ISC definitions of innovation; OR
  • Bidder has demonstrated that the proposed solution is an incremental improvement, "good engineering", or a technology that would go ahead in the normal course of product development (i.e. the next version or release).
Question 3b: Advance on State of the Art

Describe in detail the competitive advantages and level of advancement over existing technologies. Where appropriate, name existing technologies as well as potential substitutes or competitors.

To demonstrate this, proposals should include the following information:

  • Improvements (minor or major) over existing technologies or substitutes. Use direct comparison.
  • How the proposed innovation will create competitive advantages in existing market niches or market spaces.
Evaluation Schema (Mandatory Criteria — Pass/Fail + Points)

0 points/Fail:

  • The Bidder has not demonstrated that the proposed solution advances the state-of-the-art over existing technologies, including available competing solutions; OR
  • The proposed solution improves minimally upon the current state of the art, though not sufficiently enough to create competitive advantages in existing market niches; OR
  • The stated advancements are described in general terms but are not substantiated with specific, measurable evidence.

5 points/Pass:

  • The Bidder has demonstrated that the proposed solution offers one or two minor improvements to existing technologies, including available competing solutions that have potential to create competitive advantages in existing market niches.

12 points/Pass

  • The Bidder has demonstrated that the proposed solution offers three or more minor improvements to existing technologies, including available competing solutions, that together are likely to create competitive advantages in existing market niches; OR
  • The Bidder has demonstrated that the proposed solution offers one significant improvement to existing technologies that is likely to create competitive advantages in existing market niches

20 points/Pass:

  • The Bidder has demonstrated that the proposed solution offers two or more significant improvements to existing technologies, including available competing solutions that are likely to create competitive advantages in existing market niches and could define new market spaces; OR
  • The Bidder has demonstrated that the proposed solution can be considered a new benchmark of state of the art that is clearly ahead of competitors and that is likely to define new market spaces

Part 2: Point-Rated Criteria

Proposals must meet the overall minimum pass mark of 50% to be deemed responsive. Proposals that do not achieve the minimum pass mark will be declared non-responsive and given no further consideration.

Point-Rated Criteria

(Bidder's proposal to address)

Question 1b: Scope

Demonstrate the scientific and technological basis of how the proposed solution addresses the Additional Outcomes(if identified) in the Desired Outcomes section in the Challenge Notice. If no Additional Outcomes are identified in the Challenge Notice, text entered in this section will not be considered.

If no Additional Outcomes are identified in the Challenge Notice, Bidders will receive 10 points.

Evaluation Schema (Point-Rated)
  1. Insufficient or no information provided to demonstrate that the solution will address any of the Additional Outcomes. 0 points
  2. Information provided clearly demonstrates that the solution will address some (<50%) of the Additional Outcomes. 3 points
  3. Information provided clearly demonstrates that the solution will address most (50% or more) of the Additional Outcomes. 6 points
  4. Information provided clearly demonstrates that the solution will address all (100%) of the Additional Outcomes. 10 points
Question 4: Phase 1 Science and Technology (S&T) Risks

Describe potential scientific and/or technological risks to the successful development of the proof of feasibility and how they will be mitigated in Phase 1.

Evaluation Schema (Point-Rated)
  1. Insufficient or no information provided to demonstrate that the Bidder has considered potential risks and mitigation strategies and/or information provided contains significant gaps. 0 points
  2. Information provided demonstrates that the Bidder has considered some potential risks and associated mitigation strategies but there are minor gaps in risks and/or associated mitigation strategies. 5 points
  3. Information provided clearly demonstrates that the Bidder has sufficiently considered the risks and defined associated mitigation strategies. 10 points
Question 5: Phase 1 Project Plan

Demonstrate a feasible Phase 1 project plan by completing the table.

  • Indicate if any milestones and activities will be completed concurrently
  • Indicate the estimated exit TRL at the completion of Phase 1. (Drop Down Menu of the Challenge Stream Electronic Submission Form)
Evaluation Schema (Point-Rated)
  1. Insufficient or no information provided to demonstrate a feasible project plan for Phase 1 and/or the project plan exceeds the maximum duration indicated in the Challenge Notice. 0 points
  2. Project plan for Phase 1 is conceivably feasible but not clearly demonstrated and/or includes gaps. 10 points
  3. Information provided clearly demonstrates a feasible project plan for Phase 1. 20 points
Question 6: Phase 1 Project Risks

Describe potential project risks to the successful development of the proof of feasibility and how they will be mitigated in Phase 1.

Bidders should address the following risks, as applicable:

  • Human Resources
  • Financial
  • Project Management
  • Intellectual Property
  • Other project-related risks

Note to Bidders: S&T risks should not be included in this section. Question 4 addresses S&T risks.

Evaluation Schema (Point-Rated)
  1. Insufficient or no information provided to demonstrate that the Bidder has considered potential risks and mitigation strategies and/or information provided contains significant gaps. 0 points
  2. Information provided demonstrates that the Bidder has considered some potential risks and associated mitigation strategies but there are minor gaps in risks and/or associated mitigation strategies. 5 points
  3. Information provided clearly demonstrates that the Bidder has sufficiently considered the risks and defined associated mitigation strategies. 10 points
Question 7: Phase 1 Implementation Team

Demonstrate how the project implementation team has the required management and technological skill sets and experience to deliver the project plan for Phase 1 by completing the table. A member of the implementation team can have more than one role.

Evaluation Schema (Point-Rated)
  1. Insufficient or no information provided to demonstrate that the project team has the required management and technological skill sets and experience to deliver the Phase 1 project plan. 0 points
  2. Information is provided but there are minor gaps in required management and/or technological skill sets and/or experience to deliver the Phase 1 project plan. 10 points
  3. Information provided clearly demonstrates that the project team has the required management and technological skill sets and experience to deliver the Phase 1 project plan. 20 points
Question 8: Inclusivity

If your business were to receive funding from Innovative Solutions Canada, describe what actions (e.g., recruitment strategy, internships, co-op placements, etc.) might be taken in Phase 1 to support the participation of under-represented groups (e.g., women, youth, persons with disabilities, Indigenous people, visible minorities) in the research and development of the proposed solution. Each Bidder in their response to this question must focus only on describing relevant programs, policies, or initiatives that it currently has in place or would put in place to support the R&D effort in Phase 1.

Note: Do not provide any personal information of individuals employed by your company or that of your subcontractors in the response.

Evaluation Schema (Point-Rated)
  1. No description and/or concrete examples of actions provided that would be taken to encourage greater participation of under-represented groups. 0 points
  2. A description and concrete examples of actions to encourage greater participation of under-represented groups provided. 5 points
Question 9: Phase 1 Financial Proposal

Demonstrate a realistic financial proposal for the Phase 1 project plan by completing the table.

Evaluation Schema (Point-Rated)
  1. Insufficient information provided and/or information provided significantly lack credibility. Does not demonstrate a realistic financial proposal for the Phase 1 project plan. 0 points
  2. Information is provided but some costs lack credibility and/or are unclear for the Phase 1 project plan. 7.5 points
  3. Information provided contains credible elements to clearly demonstrate a realistic financial proposal for the Phase 1 project plan. 15 points
Question 10: Phase 1 Financial Controls, Tracking and Oversight

Describe the financial controls, tracking and oversight that will be used to manage the public funds throughout Phase 1. Bidders should indicate if an individual or firm will be managing the public funds and provide their credentials and/or relevant experience.

Evaluation Schema (Point-Rated)
  1. Insufficient or no information provided to demonstrate the Bidder's ability to manage public funds in Phase 1. 0 points
  2. Information provided is vague and/or contains gaps. The Bidder has some controls, tracking and/or oversight in place to manage the public funds in Phase 1. 5 points
  3. Information provided clearly demonstrates that the Bidder has strong financial controls, tracking and oversight to manage public funds in Phase 1. 10 points
Question 11: Phase 2 Overview

Demonstrate a realistic overview for the prototype development plan if selected to participate in Phase 2.

Responses should include:

  • key tasks
  • estimated cost for materials
  • human resources
  • project risks and mitigation strategies

Note: A more detailed proposal will be requested if selected to participate in Phase 2.

Evaluation Schema (Point-Rated)
  1. Insufficient or no information provided to demonstrate that the Bidder has contemplated a realistic overview for the Phase 2 prototype development. 0 points
  2. Information provided demonstrates a conceivably realistic overview for Phase 2 prototype development, however there are gaps and/or the strategy is vague. 6 points
  3. Information provided demonstrates that the Bidder has a clear and realistic overview. 12 points
Question 12: Commercialization Approach

Demonstrate a realistic overall commercialization approach/business model that can successfully take the technology/service to market, and how the technology/service will help you develop and sell other products.

Responses should include:

  • Target markets (excluding Government of Canada)
  • Non-ISC funding sources
  • Transition to a commercially-ready product or service
  • Any other indicators of commercial potential and commercial feasibility

Note: A more detailed proposal will be requested if selected to participate in Phase 2 or the Testing Stream.

Evaluation Schema (Point-Rated)
  1. Insufficient or no information provided to demonstrate that the proposed solution has commercial potential. 0 points
  2. Some information provided to demonstrate that the proposed solution has commercial potential, however there are gaps in the commercialization approach. 6 points
  3. A realistic commercialization approach is provided that demonstrates that the proposed solution has commercial potential. 12 points
Question 13: Resulting Benefits to Canada

Describe the benefits that could result from the commercialization of the proposed solution. Bidders should consider the potential benefits using the following three categories and provide justification for each claim:

  1. Innovation Benefits: Expected contribution towards the enhancement or development of new industrial or technological innovations within your firm. Responses could include: potential spillover benefits, creation of intellectual property, impact on productivity of the new technology, etc.
  2. Economic Benefits: Forecasted impact on the growth of Canadian firms, clusters and supply chains, as well as its expected benefits for Canada's workforce. Responses could include: number of jobs created, number of high-paying jobs, investment in Canada's economy, etc.
  3. Public Benefits: Expected contribution to the broader public to the degree that the solution is expected to generate social, environmental, health, security or other benefits to Canada. Responses could include: solution-related environmental benefits, solution-related accessibility benefits, and solution-related impact on Indigenous communities.
Evaluation Schema (Point-Rated)
  1. Innovation Benefits
    Benefit not identified or insufficient claim of benefit. 0 points
    Benefit has marginal increment or limited justification. 1 point
    Benefit is significant and well justified. 2 points
  2. Economic Benefits
    Benefit not identified or insufficient claim of benefit. 0 points
    Benefit has marginal increment or limited justification. 1 point
    Benefit is significant and well justified. 2 points
  3. Public Benefits
    Benefit not identified or insufficient claim of benefit. 0 points
    Benefit has marginal increment or limited justification. 1 point
    Benefit is significant and well justified. 2 points

Questions and answers

Please refer to the tender notice for this challenge on CanadaBuys.

All incoming questions regarding this specific challenge should be addressed to SIC-ISC@pwgsc.gc.ca

You can also consult the Frequently asked questions about the Innovative Solutions Canada Program.

A glossary is also available.