From 2016 to 2018, Humanitas conducted an evaluation of the National Institute of Allergy and Infectious Diseases (NIAID) Small Business Innovation Research (SBIR) and Small Business Technology Transfer Grant (STTR) Programs. We used multiple methods to assess commercialization outcomes for NIAID SBIR/STTR Phase II projects awarded in FY 2007-2012: (1) a national survey of NIAID awardees and (2) a research study using available secondary data. Humanitas administered an online survey in late 2017 to 186 awardee small businesses. For the survey, we developed and tested a survey instrument, prepared an OMB PRA clearance package, fielded the survey, analyzed survey responses, and reported findings and conclusions about commercialization of NIAID-supported projects as reported by the awardees. We also conducted a research study to determine the extent to which existing data sources can be used to evaluate the performance of the SBIR/STTR programs in fostering commercialization. For this study component, Humanitas developed and tested the data collection and analysis methodology, collected, abstracted, and analyzed data from multiple secondary sources, and reported findings and conclusions about the utility and limitations of available data, and the extent of commercialization of 196 NIAID-supported projects.
For over 15 years, Humanitas assisted HRSA’s Division of Independent Review (DIR) in administering teleconference and field reader reviews of applications for nearly all of the agency's hundreds of grant programs. Humanitas served as acting DIR review administrator to ensure an objective review process, implementing the full review process from identification of reviewers through payment of reviewer honoraria. Humanitas assisted DIR with developing policies and procedures for cost effective reviews; developing review meeting plans, policies, and manuals; managing/staffing independent review committee meetings; editing summary statements; recording application scores; preparing and tracking meeting budgets and expenditures; and providing logistical and technological support, most recently using Adobe Connect. In total, Humanitas administered logistical support for over 1,000 meetings to review approximately 49,000 grant applications, with meetings ranging in length from 1 to 12 days, and number of participants from 4 to 400 per meeting.
Since 2002, the National Cancer Institute (NCI) has used our proprietary tool ePanel to support remote evaluation of Central Institutional Review Board (CIRB) Protocols by expert reviewers. Under this ongoing contract, Humanitas provides an annual license for ePanel© for use by adult and pediatric CIRBs and enhances the application to meet changing CIRB requirements. ePanel© handles document distribution and access, records narrative reviews by individual CIRB members, and automates scoring and voting in compliance with federal privacy regulations. For each meeting, Humanitas staff provide technical assistance to CIRB members on how to access and use the web-based application, provide pre-, during, and post-meeting teleconference and web-based logistical support to CIRB members, and archive sensitive meeting results. Humanitas manages approximately six CIRB meetings a month (adult and pediatric reviews), totaling hundreds of meetings to date.
From 2006 to 2014, Humanitas conducted a series of projects to assess NLM-funded training for biomedical informatics research, and to build NLM capacity to monitor and evaluate program performance. Humanitas studied the status of and outcomes from NLM’s Informatics Research University Training Programs to determine the extent to which the programs individually and overall had achieved NLM’s stated aims. Humanitas also assessed characteristics and career outcomes for trainees supported by individual NLM fellowships or as post-docs on NLM RO1 research grants. For each project, we developed the study methodology, program logic models, and evaluation frameworks, and collected and analyzed primary and secondary data regarding program characteristics and trainee outcomes. Follow-on projects focused on updating the outcomes data, and developing strategies to monitor program performance and use evaluation as a management tool for administration of the university-based training program. We designed two types of “Scorecards” for use by NLM personnel in tracking performance data reported by individual programs in their annual and five-year progress reports.