The WorkAdvance Demonstration

Madison Strategies Towards Employment NYC NYC Tulsa Cleveland 15 Weeks 5 –12 weeks 4 –32 weeks 2 –17 weeks On site On site or at private tech. schools...

0 downloads 186 Views 834KB Size
The WorkAdvance Demonstration Testing a Sector-Based Training Program James A. Riccio 3rd MSF – CSC Social Sector Conference Singapore 13 September 2017

[email protected]

Promoting upward economic mobility for disadvantaged adults The issue • Stagnant wages for non-college educated adults since the 1970s, despite economic growth • 4-year college degree not the answer for all, but improving skills (human capital) is essential • Yet “training” often viewed as ineffective

One approach: WorkAdvance 2

WorkAdvance design and evaluation build on a history of rigorous studies • Mandatory “welfare-to-work” programs

• Case management and job placement (“work first”) • “Workfare” / community service jobs (temporary)

• Wage supplements • Post-placement coaching for retention/advancement

• Human capital components (typically via referral to other agencies): − Basic education (reading and math) − Occupational skills training 3

General lessons from careful evaluations The “good news” • Moderately successful in speeding up job entry and increasing earnings / reducing benefit receipt • Can be cost-beneficial for taxpayers and participants

But generally… • Little poverty reduction (unless wages supplemented) • Positive effects sometimes limited to certain subgroups • Little improvement in upward mobility 4

Sparse evidence proving effectiveness of “training” for disadvantaged adults Challenges • Many participants don’t complete training or receive a credential • Credential may not be valued by employers • Skills may not be in (or remain in) high demand

• Often no bridge between training completion and job in relevant field

Can sector-based strategies work better? 5

Distinguishing features of “sectoral” strategies • Training and placement into jobs in high-demand sectors with strong career pathways

• Dual-customer emphasis: Participants and employers • Substantial employer engagement at all stages of programming: − − −

Consult on curriculum design Conduct mock interviews with participants Provide hands-on training

• Provider brings together multiple employers in a field to collaborate on developing a qualified workforce 6

An earlier influential study of sector-based approaches Sectoral Employment Impact Study by Public/Private Ventures • 3-site randomized trial with experienced providers of sector programs • Strong positive effects on employment, wages, income, and other outcomes • Small samples, short-term results (2 years of follow-up), survey data only 7

The WorkAdvance Demonstration Scope

• Test of 4 sector-based programs operated by different organizations in 3 states • Longer-term and more comprehensive evaluation

Funded by a consortium

• Private foundations • Federal government (Social Innovation Fund)

Sponsored by • NYC Center for Economic Opportunity (Under former Mayor Michael Bloomberg) 8

WorkAdvance: Basic Model

Sector-Focused Training

+ Post-Placement Assistance

9

Essential elements of WorkAdvance model Total program duration: About 2 years • Careful screening to determine suitability and readiness for training • Sector-tailored career readiness preparation • Sector-focused occupational skills training • Sector-focused job development and placement • Post-placement support for retention /advancement • Some soft skills coaching throughout 10

Locations and sectors Per Scholas

St. Nicks Alliance

Madison Strategies Group

Towards Employment

New York City

New York City

Tulsa

Cleveland

Information technology

Environmental remediation

Transportation

Health care

Manufacturing

Manufacturing

11

Providers: All nonprofits, but with important differences

Per Scholas

St. Nicks Alliance

Madison Strategies Group

Towards Employment

New York City

New York City

Tulsa

Cleveland

New to Tulsa

New to manufacturing and sectoral training

Most experienced (15 years sectoral training)

Train, then place into job

Traditional workforce development Train, then place into job

Initially:

Initially:

Place first

Place first

Later:

Later:

Train first

Train first 12

Examples of occupational skills training Per Scholas

St. Nicks

Madison Strategies

Towards Employment

NYC

NYC

Tulsa

Cleveland

15 Weeks

5 – 12 weeks

4 – 32 weeks

2 – 17 weeks

At private tech. schools or community colleges

At private tech. schools or community colleges

On site • A Plus • Network Plus

On site or at private tech. schools

• Asbestos removal • Aviation • CDL with hazmat manufacturing endorsement • Comm Driver’s Lic. • Pest control • CNC machining • Diesel mechanic • Welding

• CNC machining • Welding • Phlebotomy • Patient care assist. • Nurse assistant • Medical billing 13

The Evaluation

14

Intake and random assignment June 2011-June 2013

Recruit applicants Extensive Screening: Suitability for Training Applicants agree to participate in study

Random assignment

WorkAdvance group

Control Group 15

Screening and intake process: 80% screened out (before random assignment) 100 90 80

Percentage

70

Example from Per Scholas Beginning of process

(Other sites similar)

60 50 40

• End of screening • Random assignment

30 20 10 0

16

Sample characteristics – All sites combined Sample size Total Program group Male

2,564 1,293 73%

Single, never married Age

67% 34 years

Parent, with custody

32%

Education • Less than HS diploma/GED • Only HS diploma/GED • Some college or more

6% 38% 56%

Currently employed

20%

Prior criminal conviction

24%

1717

Sample characteristics – by site Per Scholas

Madison Towards St. Nicks Strategies Employment

Sample size Total Program group

690 349

479 242

697 353

698 349

Male (%)

81

85

84

41

Some college or more (%)

63

44

58

57

Employed (%)

13

11

27

27

Prior criminal conviction (%)

10

20

40

25 18

Data sources for “impact” study UI WAGE (ADMIN.) RECORDS • Full sample (N=2,564) • Through 3 years after random assignment

YEAR 2 SURVEY • 80% response (N=2,058)

• Average follow-up at month 23

19

Impacts on completion of training in targeted sectors 100 90

Percentage of respondents

80

“Impact”

70 60 50

57.9 51.8

40

+45.9 ***

30

47.9

+31.6 ***

20

20.2 10

12.0

+31.2 *** 8.1

0

Per Scholas WA group

39.3

St. Nicks Alliance

Control group

Madison Strategies Group

+37.0 *** 10.8

Towards Employment 20

Impacts on other participation outcomes Career readiness (%) Per Scholas

89 60

Towards Employment

36

+ 29 *** 90

St. Nicks Madison Strategies

Job search assistance (%)

64

45

+26 *** 90

44

52

+50 *** 87 +35 ***

25

33

82 +46 ***

75 +30 *** 77 +52 *** 72 +39 ***

Post-employment services (%) 25

64 +39 *** 69

24

+45 *** 86

14

+72 *** 69

23

+46 *** 2121

Cost of WorkAdvance

Expenditures on program operations • $5,200 - $6,700 per participant across the four sites • Full benefit-cost analysis still to come

22

Impacts on Economic Outcomes

23

The bottom line Overall positive effect on earnings (all sites combined), but important variation across sites One site (Per Scholas): Large impacts

One site (St. Nicks): Few impacts Two sites (Madison Strategies and Towards Employment): Promising impacts (later cohort) 24

Impacts on Year 3 earnings – full sample % who earned $20,000+ per year

Average earnings + $1,864 ***

18,000

(+ 12%)

16,000 14,000 12,000

$17,481

10,000

$15,616

8,000 6,000 4,000 2,000 0

WA group

100 90 80 70 60 50 40 30 20 10 0

6.4pp*** 40.0%

33.6%

Control group 25

25

Impacts on UI employment rate over 3 years Percentage

St. Nicks

Per Scholas

100 80

60 40

Year 3: -4.6pp

Year 3: +6.8pp**

20 0 **

Percentage

100

***

** * ** *

Madison Strategies

*** *

Towards Employment

80 60

40 20

Year 3: +3.5pp

Year 3: +3.3pp

0 ** RA 2 3 4 5 6 7 8 9 10 11 12 13 WA group

** ** ** RA 2 3 4 5 6 7 8 9 10 11 12 13 Control group 26

Impacts on % employed in targeted sector in all sites (at time of survey) 100

Percent of respondents

90 80

Biggest impact

70 60

64.6 +16.5

61.1

***

50 40 30 20 10

+40.7 *** 20.4

31.8

+11.9 ***

48.2

50.1 +16.4 ***

33.6

19.9

0

Per Scholas WA group

St. Nicks Control group

Madison Strategies

Towards Employment 27

Increasing employment rate in targeted sector is not enough to increase average earnings Whether jobs in the sector pay more also matters Increase in sector employment (vs. control group)

x

Higher wages = in sector (vs. control group)

Impact on earnings

28

IMPACT ON % WITH JOB IN SECTOR

Per Scholas

HOW WELL x SECTOR PAYS

x

OVERALL = IMPACT ON EARNINGS

$$

=

$$$

=

+40.7 pp

x

St. Nicks +11.9 pp

29

Impacts on UI earnings over 3 years Per Scholas

6,000 5,000 4,000 $ 3,000 2,000 1,000 0

+27%

Year 3: - $304

Year 3: +$4,929*** ***

6,000 5,000 4,000 $ 3,000 2,000 1,000 0

St. Nicks

** ** *******************

Madison Strategies

**

Towards Employment

Year 3: +$1,371 * * RA 2 3 4 5 6 7 8 9 10 11 12 13 WA group

Year 3: +$802 * ** * ** RA 2 3 4 5 6 7 8 9 10 11 12 13 Control group 30

Impacts on % with job that offered advancement opportunities 100

“Employer offered many opportunities for advancement” (Current or most recent job – 2yr survey)

Percentage of respondents

90 80 70 60

+13.0 66.1 ***

50

+1.8 61.1

53.1

59.3

+10.7 66.1 *** 55.4

+8.4 55.6 ** 47.2

40 30 20

10 0

Per Scholas WA group

St. Nicks

Control group

Madison Strategies

Towards Employment 31

Impacts on % with job with other desirable characteristics (Current/most recent job – 2yr survey) Per Scholas Satisfied with job

***

Employer-provided health insurance

St. Madison Nicks Strategies

Towards Employment

** **

Hourly wage > $15

***

*

Regular work schedule

**

*

Regular permanent job

*

** 32

Do these factors influence effectiveness: •

The maturity/experience of the providers?



The shift to a training-first focus?



Characteristics of participants?

33

33

Impacts on Year 3 earnings, by cohort and site 25,000

20,000 15,000

Late Cohort

Early Cohort +7,123 *** -2,772

+1,074 +1,871

+3,603 *** +2,313

-801 -635

10,000

5,000 0

3434

Impacts on % who earned $20,000+ in Year 3, by cohort and site 100 90

Late Cohort

Early Cohort

80 70

60

+17.5 ***

+0.6

50 40 30

-7.1

+0.4

+9.6

+10.4 **

+10.8 **

-2.2

20 10 0

35

Impacts on Yr 3 earnings, by initial work status -327

+3,112 ***

20,000 15,000 10,000

(+20%) 18,594 18,267

+1,932 * (+14%)

18,750 15,638

15,607

13,674

5,000 0

Fully attached Semi-attached Long-term unemployed At baseline: Fully attached: Employed, or unemployed < 1month Semi-attached: Unemployed 1-6 months Long-term unemployed: Unemployed more than 6 months

36

Impacts on characteristics of current or most recent job, by work status at baseline Fully Attached

SemiLong-term Attached unemployed

Satisfied with job Saw opportunity for advancement at job

* ***

***

Had regular work schedule

*

***

Earned $20,000+ in Year 3

***

*

37

Limitations of a sector approach Not appropriate for all • Training in a just 1 sector won’t suit a diverse target group

• May need “bridge” programs to help more qualify • If can’t afford not to work, may preclude FT training

Must be nimble and have good labor mkt info •

Adapt quickly to changes in industry and employer demand

Steep learning curve for more traditional workforce providers 38

What’s next for evidence-building on sector strategies? For WorkAdvance • • •

Longer-term follow-up (5 years) Benefit-cost analysis Final report: 2020

Need better evidence on: •

Soft-skills coaching and post-employment features of sector programs (the “add-on” value)



Replication across more providers, sectors, labor markets

Other randomized trials are underway •

Results due over next few years 39

For more information Most recent MDRC reports on WorkAvance Encouraging Evidence on a Sector-Focused Advancement Strategy; Two-Year Impacts from the WorkAdvance Demonstration. (2016) By Richard Hendra, David H. Greenberg, Gayle Hamilton, Ari Oppenheim, Alexandra Pennington, Kelsey Schaberg, and Betsy L. Tessler Can Sector Strategies Promote Longer-Term Effects? ThreeYear Impacts from the WorkAdvance Demonstration. (Research Brief) (Forthcoming) By Kelsey Schaberg 40