K-ORCA: Testing a Decision Support Tool and Group Process for Selecting Interventions

BEHAVIORAL

Optimizing Responses with Collaborative Assessments-Automated (ORCA-A)

Participants will receive automated facilitation in the ORCA platform. This will entail prompts for group discussion based on group model building scripts and decision hygiene approaches proposed by Kahneman, Sibony, and Sunstein (2021). Participants will have the option of using these prompts to discuss ORCA results in group decision discussions. Optimizing Responses with Collaborative Assessments (ORCA) is a technical decision support tool based in multi-criteria decision analysis. Decision makers rate and compare evidence-based programs or other practices with ORCA, then prioritize practices for local implementation based on ORCA results. ORCA will be on a virtual platform to allow for web-based completion, automated analysis, and automated facilitation.

BEHAVIORAL

Optimizing Responses with Collaborative Assessments-Live (ORCA-L)

A facilitator will guide group decision discussions using group model building scripts and decision hygiene approaches proposed by Kahneman, Sibony, and Sunstein (2021). Facilitation will be either in-person or virtual, but occur "live" as in during real-time. Optimizing Responses with Collaborative Assessments (ORCA) is a technical decision support tool based in multi-criteria decision analysis. Decision makers rate and compare evidence-based programs or other practices with ORCA, then prioritize practices for local implementation based on ORCA results. ORCA will be on a virtual platform to allow for web-based completion, automated analysis, and automated facilitation.

Mean changes in decision assumptions, decision goals Pre and Immediately Post-intervention

Time frame: Baseline; Immediately Post-intervention

A two-item, quantitative measure will inquire about assumptions and goals in the decision to adopt an intervention. This uses a 5-point Likert scale ranging from 1 (Strongly disagree) to 5 (Strongly Agree). This outcome is part of Aim 2b: Decision quality based on intervention fit to local context.

Mean changes in decision experience quality pre-post intervention

Time frame: Baseline; Immediately Post-intervention

This will be measured across five validated, quantitative sub-scales from a measure of community partnered research: Partner Values, Synergy, Conflict and Cooperation, Participation, Participatory decision-making. Each scale has a range of 1 (Strongly Disagree) to 5 (Strongly Agree).

Mean changes in decision commitment pre-post intervention

Time frame: Baseline; Immediately Post-intervention

A two-item, Likert-scale measure regarding the decision maker's confidence in and commitment to implementing the selected intervention(s) will be gathered using a two-item measure. This uses a 5-point Likert scale ranging from 1 (Strongly disagree) to 5 (Strongly Agree). These scales relate to Aim 2a: decision experience quality.

Mean changes in perceived potential community health impact pre-post intervention

Time frame: Baseline; Immediately Post-intervention; 12-month follow-up

This will be measured across a validated, quantitative sub-scale from a measure for community partnered research: Community Health Improvement. This scale has a range of 1 (Strongly Disagree) to 5 (Strongly Agree). This is part of Aim 2b: Decision quality.

Descriptive changes in ordinal intervention rankings assisted (with ORCA) compared to unassisted (no ORCA)

Time frame: Baseline; Immediately Post-intervention

Decision makers will rank interventions in an ordinal manner at baseline using a study-specific survey. Rankings will be calculated post-intervention by ORCA as a function of completing the ORCA tool. The number of ordinal rankings will depend on the number of interventions considered, to be determined by participants. A rank of 1 = highest, followed by 2 ...n. This is part of Aim 2c: Decision quality.

Qualitatively described rationale for program adoption from pre-post intervention

Time frame: Baseline; Immediately Post-intervention

This is a study-specific, short, open-ended survey to understand decision makers' rationale for adopting programs during initial Family First Prevention Services Act decisions. There is no scale for the items, given the qualitative, open-ended nature. These are one survey of similar, but related constructs of intervention feasibility, acceptability, and appropriateness. This is part of Aim 2c: Decision quality.

Descriptive, mean group-level changes in perceived feasibility of intervention Pre-post intervention

Time frame: Baseline; Immediately Post-intervention; 12-month post-intervention

The validated, quantitative Feasibility of Intervention Measure (FIM) will be used to assess perceived implementation feasibility for each intervention considered with ORCA. Scale ranges from 1 (Completely Disagree) to 5 (Completely Agree). This outcome is part of Aim 2b regarding decision quality operationalized as intervention fit to local context.

Descriptive, mean group-level changes in perceived acceptability of intervention for implementation

Time frame: Baseline; Immediately Post-intervention; 12-month post-intervention

The validated, quantitative Acceptability of Intervention Measure (AIM) will be used to assess perceived implementation feasibility for each intervention considered with ORCA. Scale ranges from 1 (Completely Disagree) to 5 (Completely Agree). This outcome is part of Aim 2b regarding decision quality operationalized as intervention fit to local context.

Descriptive, mean group-level changes in perceived appropriateness of intervention for implementation

Time frame: Baseline; Immediately Post-intervention; 12-month post-intervention

The validated, quantitative Intervention Appropriateness Measure (IAM) will be used to assess perceived acceptability for each intervention considered with ORCA. Scale ranges from 1 (Completely Disagree) to 5 (Completely Agree). This outcome is part of Aim 2b regarding decision quality operationalized as intervention fit to local context.

Descriptive differences in implementation process fidelity (activities completed) for interventions adopted with ORCA

Time frame: Immediately Post-intervention; bi-weekly through 12-month post-intervention

A validated measure of implementation activities across 8 stages and 3 phases will be used (Universal Stages of Implementation Completion). Dates are recorded for each activity to be reported or as missing the date, but still being completed. One of the results for the calculated scores is the proportion of activities completed. A higher proportion of activities is generally positive.

Descriptive differences in implementation process fidelity (timing) for interventions adopted with ORCA

Time frame: Immediately Post-intervention; bi-weekly through 12-month post-intervention

A validated measure of implementation activities across 8 stages and 3 phases will be used (Universal Stages of Implementation Completion). Dates are recorded for each activity to be reported or as missing the date, but still being completed. One of the results for the calculated scores is the duration (time elapsed). The ideal duration depends on the proportion score.

What is a clinical trial?

A clinical trial is a research study that tests new medical treatments, drugs, devices, or procedures to determine their safety and effectiveness. Trials are carefully designed and monitored to protect participants while advancing medical knowledge.

Is it safe to participate?

Clinical trials follow strict safety guidelines and ethical standards. Trials must be reviewed and approved, and participants are closely monitored by medical professionals throughout the study. You can withdraw at any time if you choose.

Will I be compensated?

Many clinical trials offer compensation for your time, travel expenses, and inconvenience. The specific compensation varies by study and will be discussed during the screening process. All study-related medical care is typically provided at no cost to participants.

Will I receive a placebo instead of treatment?

When effective treatment exists, participants typically receive either the standard treatment plus the study intervention, or the standard treatment plus placebo. You would not be denied effective care. Placebos are primarily used when no proven treatment is available, or in addition to standard care. Your trial consent form will clearly explain what treatments you may receive.

Can I leave a trial if I change my mind?

Absolutely. Participation in clinical trials is completely voluntary. You have the right to withdraw from the study at any time, for any reason, without penalty or loss of benefits to which you are otherwise entitled.

How long does a clinical trial last?

Trial duration varies widely depending on the study design and purpose. Some trials last just a few weeks, while others may continue for months or years. The study coordinator will provide specific timeline information during your screening call.