WEBVTT

00:41:45.369 --> 00:41:48.172
GOOD AFTERNOON

00:41:48.239 --> 00:41:49.240
EVERYBODY. WE'RE GOING TO CALL

00:41:49.306 --> 00:41:51.308
THIS SPECIAL INFORMATIONAL

00:41:51.375 --> 00:41:53.110
HEARING OF THE PRIVACY AND

00:41:53.177 --> 00:41:54.578
CONSUMER PROTECTION COMMITTEE

00:41:54.645 --> 00:41:57.414
TO ORDER. I WANT TO BEGIN BY

00:41:57.481 --> 00:41:59.750
THANKING THE STAFF, AS ALWAYS,

00:41:59.817 --> 00:42:01.118
FOR THE INCREDIBLY HARD WORK IN

00:42:01.185 --> 00:42:02.887
PUTTING THIS TOGETHER AND THE

00:42:02.953 --> 00:42:03.988
BRILLIANT BACKGROUNDER. IF YOU

00:42:04.054 --> 00:42:05.222
DIDN'T GET A CHANCE TO READ IT,

00:42:05.289 --> 00:42:06.991
I HIGHLY RECOMMEND IT. IT'S

00:42:07.057 --> 00:42:09.326
GOOD READING, AND I WANT TO

00:42:09.393 --> 00:42:12.229
THANK THE SPEAKER AND HIS TEAM

00:42:12.296 --> 00:42:13.497
FOR A COUPLE OF THINGS. ONE IS

00:42:13.564 --> 00:42:15.566
THE OPPORTUNITY TO CHAIR THIS

00:42:15.633 --> 00:42:16.600
COMMITTEE AT THIS MOMENT IN

00:42:16.667 --> 00:42:18.702
TIME AND LEAD OUR HOUSE IN, I

00:42:18.769 --> 00:42:20.471
THINK, WHAT IS AN INCREDIBLY

00:42:20.537 --> 00:42:22.273
CONSEQUENTIAL MOMENT IN

00:42:22.339 --> 00:42:23.407
CALIFORNIA'S HISTORY AS WE

00:42:23.474 --> 00:42:25.509
START TO FACE THE REALITY OF

00:42:25.576 --> 00:42:26.644
ARTIFICIAL INTELLIGENCE AND

00:42:26.710 --> 00:42:28.812
THESE FRONTIER MODELS AND THEIR

00:42:28.879 --> 00:42:29.847
INCREDIBLE CAPABILITIES THAT

00:42:29.914 --> 00:42:31.348
ARE BEING BUILT IN OUR

00:42:31.415 --> 00:42:32.383
BACKYARDS, WHILE ENSURING THE

00:42:32.449 --> 00:42:33.550
SAFETY OF OUR COMMUNITIES THE

00:42:33.617 --> 00:42:35.686
WAY WE REPRESENT EVERY DAY, AND

00:42:35.753 --> 00:42:37.421
FOR SUPPORTING THIS HEARING

00:42:37.488 --> 00:42:38.822
TODAY SO THAT WE CAN HAVE AN

00:42:38.889 --> 00:42:39.790
IMPORTANT CONVERSATION ABOUT

00:42:39.857 --> 00:42:40.891
THE STATE OF THE LAW IN

00:42:40.958 --> 00:42:42.726
CALIFORNIA TODAY AND WHERE IT

00:42:42.793 --> 00:42:43.928
NEEDS TO GO TO REALLY ENSURE

00:42:43.994 --> 00:42:45.362
THE SAFETY OF OUR COMMUNITIES.

00:42:45.429 --> 00:42:47.531
CALIFORNIA HAS, OVER THE PAST

00:42:47.598 --> 00:42:49.300
COUPLE OF YEARS, LED IN THE

00:42:49.366 --> 00:42:51.835
ARTIFICIAL INTELLIGENCE POLICY

00:42:51.902 --> 00:42:53.003
REALM. WE HAVE PASSED SOME OF

00:42:53.070 --> 00:42:54.104
THE STRONGEST IN THE NATION

00:42:54.171 --> 00:42:55.940
LAWS. WHETHER IT WAS THE BILL

00:42:56.006 --> 00:42:57.074
WE'LL BE TALKING ABOUT TODAY,

00:42:57.141 --> 00:42:58.909
SB 53, WHICH WAS THE FIRST IN

00:42:58.976 --> 00:43:00.110
THE NATION TO ADDRESS THE

00:43:00.177 --> 00:43:01.946
CATASTROPHIC RISKS. AND FRANKLY,

00:43:02.012 --> 00:43:04.081
SENATOR WIENER, WHO WILL BE ON

00:43:04.148 --> 00:43:06.216
SHORTLY, WAS AHEAD OF HIS TIME.

00:43:06.283 --> 00:43:08.319
HE FORESAW RISKS THAT AT THE

00:43:08.385 --> 00:43:09.453
TIME I WAS BEING TOLD BY

00:43:09.520 --> 00:43:10.888
EXPERTS, WE'RE A DECADE OUT.

00:43:10.955 --> 00:43:13.390
AND HE PUT ALL OF HIS ENERGY

00:43:13.457 --> 00:43:15.592
AND HIS POLITICAL POWER INTO

00:43:15.659 --> 00:43:16.994
ENSURING THAT OUR COMMUNITIES

00:43:17.061 --> 00:43:19.530
HAD THE SAFETY THAT THEY NEEDED.

00:43:19.596 --> 00:43:22.499
AND HE TRIED ONCE WITH SB 1047,

00:43:22.566 --> 00:43:25.269
THAT BILL DID NOT MAKE IT TO

00:43:25.336 --> 00:43:27.004
LAW. BUT THEN HE TRIED AGAIN

00:43:27.071 --> 00:43:29.306
WITH SB 53. AND FOR THE FIRST

00:43:29.373 --> 00:43:31.208
TIME IN THE NATION, THERE WAS A

00:43:31.275 --> 00:43:32.176
REPORTING REGIME AROUND WHAT

00:43:32.242 --> 00:43:33.577
COULD BE CONSEQUENTIAL,

00:43:33.644 --> 00:43:34.678
CATASTROPHIC RISK. AND WE KNOW

00:43:34.745 --> 00:43:35.846
TODAY THAT IT ISN'T COVERING A

00:43:35.913 --> 00:43:37.748
LOT OF WHAT WE ARE SEEING, AND

00:43:37.815 --> 00:43:39.650
WE NEED TO REVISIT WHAT WE NEED

00:43:39.717 --> 00:43:40.951
TO DO TO KEEP COMMUNITIES SAFE.

00:43:41.018 --> 00:43:42.619
BUT IT REALLY PUT CALIFORNIA IN

00:43:42.686 --> 00:43:43.787
POSITION WE'RE IN TODAY. SO I

00:43:43.854 --> 00:43:45.622
WANT TO EXPRESS MY GRATITUDE TO

00:43:45.689 --> 00:43:47.624
SENATOR WIENER. LAST YEAR, WE

00:43:47.691 --> 00:43:49.793
PASSED THE STRONGEST AI BILL IN

00:43:49.860 --> 00:43:51.595
THE NATION. AS I SEE IT, AROUND

00:43:51.662 --> 00:43:53.731
KIDS SAFETY. FOR THE FIRST TIME,

00:43:53.797 --> 00:43:55.532
CALIFORNIA SAID YOU CANNOT

00:43:55.599 --> 00:43:57.768
RELEASE AI CHATBOTS TO CHILDREN

00:43:57.835 --> 00:43:59.169
UNLESS YOU'VE MET CERTAIN

00:43:59.236 --> 00:44:00.204
THRESHOLDS. WE ARE GOING TO

00:44:00.270 --> 00:44:02.006
DEMAND SAFETY FOR CALIFORNIA'S

00:44:02.072 --> 00:44:03.707
KIDS, AND I BELIEVE IT IS

00:44:03.774 --> 00:44:04.875
ACTUALLY A MODEL FOR THE WAY WE

00:44:04.942 --> 00:44:06.744
NEED TO BE THINKING ABOUT AI

00:44:06.810 --> 00:44:08.112
SAFETY. IT ISN'T ACCEPTABLE TO

00:44:08.178 --> 00:44:10.080
JUST TO ALLOW FOR THESE MODELS

00:44:10.147 --> 00:44:11.982
TO BE PUT INTO THE WILD AND

00:44:12.049 --> 00:44:12.983
THEN ACCEPT THE CONSEQUENCES.

00:44:13.050 --> 00:44:14.184
WE NEED TO MAKE SURE THAT WE

00:44:14.251 --> 00:44:15.953
ARE SETTING STANDARDS BY WHICH

00:44:16.020 --> 00:44:17.154
OUR COMPANIES CAN TRY TO ENSURE

00:44:17.221 --> 00:44:18.288
THE SAFETY OF OUR COMMUNITIES,

00:44:18.355 --> 00:44:20.024
AND THAT IS THE CONVERSATION WE

00:44:20.090 --> 00:44:21.959
WILL HAVE TODAY. SB 53 IS A

00:44:22.026 --> 00:44:22.993
STARTING POINT, BUT TODAY I

00:44:23.060 --> 00:44:24.928
THINK WE WILL SEE THAT IT IS

00:44:24.995 --> 00:44:26.196
NOT THE END POINT BY ANY MEANS.

00:44:26.263 --> 00:44:28.165
WE HAVE A LOT OF WORK AHEAD OF

00:44:28.232 --> 00:44:30.200
US, AND I WANT TO THANK ALL THE

00:44:30.267 --> 00:44:31.969
PANELISTS THAT ARE HERE TODAY

00:44:32.036 --> 00:44:33.137
TO ADDRESS WHAT CAN BE DONE IN

00:44:33.203 --> 00:44:35.105
THE YEAR AHEAD. WE DO PLAN TO

00:44:35.172 --> 00:44:37.041
TACKLE THIS AS SOON AS WE ARE

00:44:37.107 --> 00:44:38.142
BACK IN SESSION. WE BELIEVE

00:44:38.208 --> 00:44:40.077
THAT THERE IS A LOT OF WORK TO

00:44:40.144 --> 00:44:42.112
BE DONE, AND WE WILL BE WORKING

00:44:42.179 --> 00:44:43.247
TOGETHER AS A CAUCUS TO ENSURE

00:44:43.313 --> 00:44:45.015
THAT WE ARE PUTTING FORTH A

00:44:45.082 --> 00:44:46.083
PACKAGE OF BILLS THAT IS

00:44:46.150 --> 00:44:47.217
COMPREHENSIVE IN ITS PROTECTION

00:44:47.284 --> 00:44:48.952
OF OUR COMMUNITIES, AND I WANT

00:44:49.019 --> 00:44:50.621
TO THANK THE SPEAKER FOR HIS

00:44:50.687 --> 00:44:52.823
PARTNERSHIP IN THAT AS WELL. SO

00:44:52.890 --> 00:44:54.558
WITH THAT, I'M EXCITED FOR THE

00:44:54.625 --> 00:44:55.692
CONVERSATION. I THINK WE HAVE A

00:44:55.759 --> 00:44:57.628
LOT TO DISCUSS, AND I WANT TO

00:44:57.694 --> 00:44:59.563
GET TO IT. BUT FIRST, I KNOW

00:44:59.630 --> 00:45:00.731
THE SPEAKER WANTS TO MAKE SOME

00:45:00.798 --> 00:45:03.133
OPENING REMARKS. YEAH, WELL, I

00:45:03.200 --> 00:45:06.103
WANT TO I APPRECIATE THAT

00:45:06.170 --> 00:45:06.904
INTRODUCTION, MADAM CHAIR, AND

00:45:06.970 --> 00:45:08.605
THANK YOU FOR YOUR WORK. IT'S

00:45:08.672 --> 00:45:11.542
EXCITING TO SEE ALL OF YOU HERE.

00:45:11.608 --> 00:45:12.976
AND IT SEEMS LIKE EVERYONE IS

00:45:13.043 --> 00:45:15.245
EAGER TO GET BACK TO WORK. SO

00:45:15.312 --> 00:45:17.281
IT'S GOOD. IT'S GOOD, YOU KNOW.

00:45:17.347 --> 00:45:18.782
BUT I AS WELL WANT TO THANK

00:45:18.849 --> 00:45:21.185
STAFF FOR THE TIME IN IN

00:45:21.251 --> 00:45:22.986
PLANNING ALL THE MATERIALS

00:45:23.053 --> 00:45:26.557
REALLY, YOU KNOW, THOROUGH

00:45:26.623 --> 00:45:28.058
PACKAGE OF INFORMATION. SO

00:45:28.125 --> 00:45:29.927
THANK YOU ALL FOR THAT. I WANT

00:45:29.993 --> 00:45:31.695
TO THANK STAKEHOLDERS WHO ARE

00:45:31.762 --> 00:45:34.064
HERE AS WELL. BUT BUT LOOK, I

00:45:34.131 --> 00:45:35.999
WANT TO BE VERY CLEAR ABOUT ONE

00:45:36.066 --> 00:45:38.836
THING. CALIFORNIA IS THE GLOBAL

00:45:38.902 --> 00:45:40.337
LEADER IN AI. WE'RE THE GLOBAL

00:45:40.404 --> 00:45:42.039
LEADER IN ARTIFICIAL

00:45:42.106 --> 00:45:43.540
INTELLIGENCE. AND WITH THAT

00:45:43.607 --> 00:45:46.343
LEADERSHIP COMES AN ENORMOUS

00:45:46.410 --> 00:45:48.545
RESPONSIBILITY. AND, YOU KNOW,

00:45:48.612 --> 00:45:50.747
THERE'S NO QUESTION THAT AI HAS

00:45:50.814 --> 00:45:53.984
THE POTENTIAL TO TRANSFORM OUR

00:45:54.051 --> 00:45:56.553
ECONOMY, TO IMPROVE LIVES, AND

00:45:56.620 --> 00:45:58.288
TO CREATE OPPORTUNITIES THAT WE

00:45:58.355 --> 00:46:00.057
COULD SCARCELY IMAGINE JUST A

00:46:00.124 --> 00:46:03.026
FEW YEARS AGO. BUT LET'S BE

00:46:03.093 --> 00:46:05.629
EQUALLY CLEAR INNOVATION CANNOT

00:46:05.696 --> 00:46:07.998
COME AT THE EXPENSE OF PUBLIC

00:46:08.065 --> 00:46:13.070
SAFETY. YOU KNOW, THE THE, THE,

00:46:13.137 --> 00:46:14.805
THE TECHNOLOGICAL, THE

00:46:14.872 --> 00:46:16.640
TECHNOLOGICAL PROGRESS. IT

00:46:16.707 --> 00:46:18.775
CANNOT MEAN ABANDONING OUR

00:46:18.842 --> 00:46:20.244
RESPONSIBILITY TO PROTECT THE

00:46:20.310 --> 00:46:23.147
PEOPLE WE SERVE OUR CHILDREN,

00:46:23.213 --> 00:46:25.315
OUR WORKERS, OUR FAMILIES.

00:46:25.382 --> 00:46:26.717
EVERY PERSON WHO CALLS

00:46:26.783 --> 00:46:28.652
CALIFORNIA HOME. AND THAT'S WHY

00:46:28.719 --> 00:46:30.354
THE WORK OF THIS LEGISLATURE IS

00:46:30.420 --> 00:46:31.788
INCREDIBLY IMPORTANT. YOU KNOW,

00:46:31.855 --> 00:46:33.257
LAST YEAR, OUR STATE TOOK A

00:46:33.323 --> 00:46:34.658
MAJOR STEP FORWARD, AS WAS

00:46:34.725 --> 00:46:36.493
MENTIONED BY THE CHAIR WITH

00:46:36.560 --> 00:46:39.696
SENATE BILL 53, WHICH, AS SHE

00:46:39.763 --> 00:46:40.397
MENTIONED, ESTABLISHED THE

00:46:40.464 --> 00:46:42.633
NATION'S FIRST TRANSPARENCY

00:46:42.699 --> 00:46:44.368
FRAMEWORK FOR FRONTIER AI

00:46:44.434 --> 00:46:45.802
DEVELOPERS. THAT WAS AN

00:46:45.869 --> 00:46:48.338
IMPORTANT ACCOMPLISHMENT. BUT A

00:46:48.405 --> 00:46:50.507
PASSING LEGISLATION IS NOT THE

00:46:50.574 --> 00:46:53.010
FINISH LINE. IT'S THE BEGINNING

00:46:53.076 --> 00:46:55.179
OF OUR RESPONSIBILITY AS A

00:46:55.245 --> 00:46:56.246
LEGISLATURE TO MAKE SURE THE

00:46:56.313 --> 00:47:00.317
LAW ACTUALLY WORKS. AND THAT IS

00:47:00.384 --> 00:47:02.352
PRECISELY WHY WE ARE HERE TODAY

00:47:02.419 --> 00:47:03.587
TO HAVE THIS HEARING, BECAUSE

00:47:03.654 --> 00:47:05.255
WE NEED TO KNOW WHETHER THE

00:47:05.322 --> 00:47:06.690
SAFEGUARDS WE'VE ESTABLISHED

00:47:06.757 --> 00:47:08.525
ARE ACTUALLY WORKING. WE NEED

00:47:08.592 --> 00:47:09.927
TO KNOW WHETHER COMPANIES ARE

00:47:09.993 --> 00:47:11.795
BEING TRANSPARENT ABOUT THE

00:47:11.862 --> 00:47:13.864
RISKS THEY'RE CREATING. AND WE

00:47:13.931 --> 00:47:14.932
NEED TO KNOW WHETHER

00:47:14.998 --> 00:47:16.800
INDEPENDENT EXPERTS AND

00:47:16.867 --> 00:47:18.202
REGULATORS HAVE THE ACCESS AND

00:47:18.268 --> 00:47:20.737
THE INFORMATION NECESSARY TO

00:47:20.804 --> 00:47:22.539
HOLD THOSE COMPANIES

00:47:22.606 --> 00:47:24.675
ACCOUNTABLE. THE RECENT HUGGING

00:47:24.741 --> 00:47:26.610
FACE INCIDENT UNDERSCORES WHY

00:47:26.677 --> 00:47:30.514
THESE QUESTIONS MATTER. BECAUSE

00:47:30.581 --> 00:47:32.249
TRANSPARENCY WITHOUT

00:47:32.316 --> 00:47:34.785
ACCOUNTABILITY IS NOT ENOUGH,

00:47:34.851 --> 00:47:36.253
AND OVERSIGHT WITHOUT

00:47:36.320 --> 00:47:39.489
MEANINGFUL ACCESS IS NOT REAL

00:47:39.556 --> 00:47:42.059
OVERSIGHT. LOOK, I BELIEVE THAT

00:47:42.125 --> 00:47:43.427
HERE IN CALIFORNIA, WE CAN LEAD

00:47:43.493 --> 00:47:45.662
THE WORLD IN AI INNOVATION, BUT

00:47:45.729 --> 00:47:48.131
WE CAN ALSO LEAD THE NATION IN

00:47:48.198 --> 00:47:50.400
PROTECTING THE PUBLIC. THESE

00:47:50.467 --> 00:47:52.402
ARE NOT SOME COMPETING

00:47:52.469 --> 00:47:53.870
PRIORITIES, BUT ENSURING THAT

00:47:53.937 --> 00:47:56.039
WE GET THIS RIGHT REQUIRES MORE

00:47:56.106 --> 00:47:58.575
THAN JUST GOOD INTENTIONS. IT

00:47:58.642 --> 00:48:00.410
REQUIRES VIGILANCE. IT REQUIRES

00:48:00.477 --> 00:48:02.112
INDEPENDENT SCRUTINY AND A

00:48:02.179 --> 00:48:03.614
WILLINGNESS TO STRENGTHEN OUR

00:48:03.680 --> 00:48:05.682
LAWS WHEN CIRCUMSTANCES DEMAND

00:48:05.749 --> 00:48:08.652
IT. AS WE PREPARE FOR THIS NEXT

00:48:08.719 --> 00:48:11.888
YEAR IN 2027, I WANT US TO BE

00:48:11.955 --> 00:48:14.057
AS PREPARED AS POSSIBLE TO DO

00:48:14.124 --> 00:48:16.827
EXACTLY THAT. YOU KNOW, WE

00:48:16.893 --> 00:48:17.995
WELCOME PARTNERSHIP WITH

00:48:18.061 --> 00:48:20.097
INDUSTRY LEADERS, WITH

00:48:20.163 --> 00:48:21.098
RESEARCHERS AND WITH THE

00:48:21.164 --> 00:48:22.966
FEDERAL GOVERNMENT, BUT HAVE TO

00:48:23.033 --> 00:48:24.768
BE ABSOLUTELY CLEAR THAT

00:48:24.835 --> 00:48:26.470
CALIFORNIA WILL NOT SURRENDER

00:48:26.536 --> 00:48:29.172
ITS RESPONSIBILITY TO PROTECT

00:48:29.239 --> 00:48:30.607
CALIFORNIANS. FIRST AND

00:48:30.674 --> 00:48:31.842
FOREMOST, FEDERAL STANDARDS

00:48:31.908 --> 00:48:33.777
SHOULD ESTABLISH A FLOOR FOR

00:48:33.844 --> 00:48:35.912
PUBLIC SAFETY, NOT A CEILING ON

00:48:35.979 --> 00:48:38.882
WHAT STATES CAN DO, YOU KNOW,

00:48:38.949 --> 00:48:40.550
AND WE CAN'T AFFORD TO WAIT FOR

00:48:40.617 --> 00:48:42.819
THE NEXT MAJOR INCIDENT BEFORE

00:48:42.886 --> 00:48:44.288
ASKING WHETHER OUR LAWS ARE

00:48:44.354 --> 00:48:45.722
STRONG ENOUGH. THAT'S WHAT THIS

00:48:45.789 --> 00:48:47.124
HEARING IS ALL ABOUT HERE TODAY,

00:48:47.190 --> 00:48:48.191
ASKING TOUGH QUESTIONS,

00:48:48.258 --> 00:48:49.326
ENSURING THAT WE DEMAND

00:48:49.393 --> 00:48:51.094
MEANINGFUL ANSWERS, AND MAKING

00:48:51.161 --> 00:48:52.462
SURE THAT THE PROTECTIONS WE

00:48:52.529 --> 00:48:54.398
PUT INTO LAW DELIVER REAL

00:48:54.464 --> 00:48:56.166
RESULTS FOR CALIFORNIANS. AND

00:48:56.233 --> 00:48:58.268
SO, BECAUSE WE ALL KNOW THAT,

00:48:58.335 --> 00:48:59.369
YOU KNOW, THIS TECHNOLOGY WILL

00:48:59.436 --> 00:49:00.570
CONTINUE TO EVOLVE, WE KNOW

00:49:00.637 --> 00:49:02.172
THAT THE RISKS WILL CONTINUE TO

00:49:02.239 --> 00:49:03.373
EVOLVE, AND OUR RESPONSIBILITY

00:49:03.440 --> 00:49:05.142
TO PROTECT OUR RESIDENTS AND

00:49:05.208 --> 00:49:06.643
THE PUBLIC MUST NEVER, EVER

00:49:06.710 --> 00:49:07.911
FALL BEHIND. SO WITH THAT,

00:49:07.978 --> 00:49:09.546
THANK YOU, MADAM CHAIR. THANK

00:49:09.613 --> 00:49:11.448
YOU, MR. SPEAKER, AND I WANT TO

00:49:11.515 --> 00:49:12.616
THANK ALL OF THE MEMBERS WHO

00:49:12.683 --> 00:49:13.650
ARE HERE. IF ANYONE IS

00:49:13.717 --> 00:49:15.585
WONDERING IF THIS IS A PRIORITY

00:49:15.652 --> 00:49:16.653
FOR OUR CAUCUS, I THINK YOUR

00:49:16.720 --> 00:49:18.555
ANSWER IS CLEAR FROM THE NUMBER

00:49:18.622 --> 00:49:19.690
OF PEOPLE WHO SHOWED UP FROM

00:49:19.756 --> 00:49:21.525
ACROSS THE STATE TODAY TO HAVE

00:49:21.591 --> 00:49:22.592
THIS CONVERSATION. SO I THANK

00:49:22.659 --> 00:49:24.561
ALL OF YOU. I KNOW SOME OF YOU

00:49:24.628 --> 00:49:26.563
CAME FROM THIS FAR IN THE STATE

00:49:26.630 --> 00:49:27.764
AS YOU COULD POSSIBLY COME, SAN

00:49:27.831 --> 00:49:29.566
DIEGO. SO THANK YOU. WITH THAT,

00:49:29.633 --> 00:49:31.468
WE ARE GOING TO HAVE TIME FOR

00:49:31.535 --> 00:49:32.536
MEMBERS TO MAKE COMMENTS, BUT

00:49:32.602 --> 00:49:33.704
SENATOR WIENER IS JOINING US TO

00:49:33.770 --> 00:49:35.505
MAKE SOME COMMENTS ON SB 53.

00:49:35.572 --> 00:49:37.407
AND HE HAS A HARD STOP, SO

00:49:37.474 --> 00:49:38.608
WE'RE GOING TO LET HIM JOIN US

00:49:38.675 --> 00:49:40.444
VIRTUALLY PRIOR TO THAT. SO

00:49:40.510 --> 00:49:41.545
MAYBE SENATOR WIENER IS GOING

00:49:41.611 --> 00:49:42.679
TO APPEAR ON THE SCREEN, SEE

00:49:42.746 --> 00:49:54.891
WHAT HAPPENS. THERE HE IS.

00:49:54.958 --> 00:49:57.361
MAYBE. GREAT. YEAH. THERE YOU

00:49:57.427 --> 00:49:58.462
ARE. HI, SENATOR. THE FLOOR IS

00:49:58.528 --> 00:50:00.697
YOURS. THANK YOU. HI. CAN YOU

00:50:00.764 --> 00:50:03.800
HEAR ME? YES, WE'VE GOT YOU.

00:50:03.867 --> 00:50:05.635
GREAT. THANK YOU VERY MUCH,

00:50:05.702 --> 00:50:08.238
MADAM CHAIR. AND THANK YOU, MR.

00:50:08.305 --> 00:50:09.606
SPEAKER, FOR CONVENING THIS

00:50:09.673 --> 00:50:13.543
HEARING TODAY AND REALLY MAKING

00:50:13.610 --> 00:50:15.412
CLEAR THAT THIS IS IMPORTANT,

00:50:15.479 --> 00:50:17.914
THAT THIS MATTERS. AND I'M VERY

00:50:17.981 --> 00:50:18.615
APPRECIATIVE OF, MADAM CHAIR.

00:50:18.682 --> 00:50:20.217
IN PARTICULAR, I WANT TO THANK

00:50:20.283 --> 00:50:23.220
YOU FOR YOUR PARTNERSHIP ON SO

00:50:23.286 --> 00:50:25.856
MUCH OF THIS WORK. WE, YOU AND

00:50:25.922 --> 00:50:28.725
I HAVE WORKED INTENSIVELY

00:50:28.792 --> 00:50:30.894
TOGETHER FOR MULTIPLE YEARS NOW,

00:50:30.961 --> 00:50:34.831
GOING BACK TO 2024, AND YOU

00:50:34.898 --> 00:50:36.266
HAVE PLAYED A CRITICAL ROLE IN

00:50:36.333 --> 00:50:38.769
WORKING WITH US TO HELP CRAFT

00:50:38.835 --> 00:50:41.271
GOOD POLICY AROUND AI

00:50:41.338 --> 00:50:42.806
REGULATION WHEN IT COMES TO

00:50:42.873 --> 00:50:44.875
SAFETY AND CATASTROPHIC RISK.

00:50:44.941 --> 00:50:46.109
AND I WANT TO JUST THANK YOU

00:50:46.176 --> 00:50:48.011
FOR YOUR WORK WITH US AND ALSO

00:50:48.078 --> 00:50:49.079
YOUR WORK ON YOUR OWN

00:50:49.146 --> 00:50:50.213
LEGISLATION, WHICH HAS BEEN

00:50:50.280 --> 00:50:52.249
INCREDIBLY IMPORTANT. AND THANK

00:50:52.315 --> 00:50:53.550
YOU FOR GIVING ME THE

00:50:53.617 --> 00:50:54.985
OPPORTUNITY TO SPEAK TODAY ON

00:50:55.051 --> 00:50:56.453
THIS OUTCOMES REVIEW AROUND

00:50:56.520 --> 00:50:58.555
SENATE BILL 53, WHICH, AS YOU

00:50:58.622 --> 00:51:00.157
MENTIONED, IS THE FIRST LAW IN

00:51:00.223 --> 00:51:04.661
THE NATION TO REGULATE AI

00:51:04.728 --> 00:51:06.863
AROUND SAFETY AND TO ESTABLISH

00:51:06.930 --> 00:51:08.031
PROTOCOLS TO ADDRESS THE RISK

00:51:08.098 --> 00:51:10.400
OF CATASTROPHIC HARM STEMMING

00:51:10.467 --> 00:51:12.636
FROM MODELS AT THE FRONTIER OF

00:51:12.702 --> 00:51:15.105
AI DEVELOPMENT. AND I'M PROUD

00:51:15.172 --> 00:51:18.008
TO HAVE AUTHORED THAT LAW, BUT

00:51:18.074 --> 00:51:19.910
I IT WAS MY NAME WAS AT THE TOP,

00:51:19.976 --> 00:51:22.345
BUT IT WAS A HUGE COALITION

00:51:22.412 --> 00:51:24.147
EFFORT. AND I'M PROUD OF THE

00:51:24.214 --> 00:51:25.549
COALITION THAT WE BUILT TO HELP

00:51:25.615 --> 00:51:28.518
MAKE CALIFORNIA THE FIRST AND

00:51:28.585 --> 00:51:30.987
TO CONTINUE THE WORK. THE RISK

00:51:31.054 --> 00:51:32.456
THAT ARTIFICIAL INTELLIGENCE

00:51:32.522 --> 00:51:34.224
POSES TO OUR SAFETY IS AN ISSUE

00:51:34.291 --> 00:51:35.325
OF CRITICAL IMPORTANCE, NOT

00:51:35.392 --> 00:51:37.060
JUST FOR CALIFORNIA, BUT FOR

00:51:37.127 --> 00:51:39.629
THE ENTIRE WORLD. OUR STATE, AS

00:51:39.696 --> 00:51:41.765
THE SPEAKER MENTIONED, IS THE

00:51:41.832 --> 00:51:43.200
BEATING HEART OF TECH

00:51:43.266 --> 00:51:45.735
INNOVATION. AND IF I MAY SAY,

00:51:45.802 --> 00:51:47.504
MY HOME CITY OF SAN FRANCISCO

00:51:47.571 --> 00:51:49.272
IS THE BEATING HEART OF THE

00:51:49.339 --> 00:51:51.842
BEATING HEART OF OF AI

00:51:51.908 --> 00:51:53.577
INNOVATION. AND BEING A GLOBAL

00:51:53.643 --> 00:51:55.779
LEADER MEANS MODELING

00:51:55.846 --> 00:51:57.247
RESPONSIBILITY AND BEST

00:51:57.314 --> 00:51:59.749
PRACTICES. TIME AND TIME AGAIN,

00:51:59.816 --> 00:52:01.184
CALIFORNIA HAS PRODUCED

00:52:01.251 --> 00:52:02.986
POWERFUL NEW TECHNOLOGIES THAT

00:52:03.053 --> 00:52:05.822
HAVE RAPIDLY RESHAPED OUR WORLD,

00:52:05.889 --> 00:52:07.357
ONLY TO FIND THAT THE IMPACTS

00:52:07.424 --> 00:52:09.459
OF THESE TRANSFORMATIVE

00:52:09.526 --> 00:52:12.062
TECHNOLOGIES ARE BEING IGNORED,

00:52:12.128 --> 00:52:14.898
PARTICULARLY IN WASHINGTON, DC.

00:52:14.965 --> 00:52:17.067
AND AS A RESULT, CALIFORNIA HAS

00:52:17.133 --> 00:52:20.670
STEPPED UP THE LIST OF WELL

00:52:20.737 --> 00:52:21.771
DOCUMENTED HARMS OF SOCIAL

00:52:21.838 --> 00:52:23.874
MEDIA HAS INFLICTED ON OUR

00:52:23.940 --> 00:52:25.675
WORLD. WORLD GROWS LARGER EACH

00:52:25.742 --> 00:52:28.211
YEAR, AND WE ALL SEE IT IN OUR

00:52:28.278 --> 00:52:30.714
LIVES. AND AS POLICY MAKERS.

00:52:30.780 --> 00:52:32.549
AND WE HAVE SEEN CONGRESS

00:52:32.616 --> 00:52:34.451
REALLY MAKE VERY LITTLE EFFORT

00:52:34.518 --> 00:52:36.520
TO REIN IN OR REGULATE SOCIAL

00:52:36.586 --> 00:52:38.288
MEDIA. THE SAME WITH DATA

00:52:38.355 --> 00:52:39.389
PRIVACY. THERE'S NO FEDERAL

00:52:39.456 --> 00:52:41.925
DATA PRIVACY LAW IN THE YEAR

00:52:41.992 --> 00:52:43.693
2026, WHICH IS SORT OF

00:52:43.760 --> 00:52:44.194
UNBELIEVABLE. THERE'S NO

00:52:44.261 --> 00:52:46.229
FEDERAL NET NEUTRALITY LAW IN

00:52:46.296 --> 00:52:48.765
EACH OF THESE CASES. CALIFORNIA

00:52:48.832 --> 00:52:50.534
HAS STEPPED UP. WE PASSED A

00:52:50.600 --> 00:52:52.669
DATA PRIVACY LAW. WE HAVE

00:52:52.736 --> 00:52:54.104
REPEATEDLY MOVED TO REGULATE

00:52:54.170 --> 00:52:56.373
SOCIAL MEDIA TO DEAL WITH THE

00:52:56.439 --> 00:52:57.574
HARMS THAT IT IS INFLICTING ON

00:52:57.641 --> 00:52:59.442
OUR COMMUNITIES. WE PASSED A

00:52:59.509 --> 00:53:02.045
NET NEUTRALITY LAW, AND I'M

00:53:02.112 --> 00:53:03.513
REALLY PROUD OF THE WORK THAT

00:53:03.580 --> 00:53:06.049
WE HAVE DONE AS A STATE. BUT

00:53:06.116 --> 00:53:09.252
THE FACT REMAINS THAT THIS

00:53:09.319 --> 00:53:10.987
COUNTRY AS A WHOLE HAS NOT THE

00:53:11.054 --> 00:53:12.455
BEST TRACK RECORD OF STAYING

00:53:12.522 --> 00:53:14.291
AHEAD OF NEW TECHNOLOGIES WHEN

00:53:14.357 --> 00:53:16.192
IT COMES TO THE RISKS AND HARMS

00:53:16.259 --> 00:53:18.228
THAT THOSE TECHNOLOGIES CAN

00:53:18.295 --> 00:53:19.996
CAUSE. THERE ARE BENEFITS, BUT

00:53:20.063 --> 00:53:22.198
THERE ARE ALWAYS RISKS. AND WE

00:53:22.265 --> 00:53:23.867
HAVE NOT, AS A COUNTRY, MANAGED

00:53:23.934 --> 00:53:26.469
THOSE RISKS WELL FOR NEW

00:53:26.536 --> 00:53:31.942
TECHNOLOGY IN 2023, I WHEN A

00:53:32.008 --> 00:53:33.677
NUMBER OF MY CONSTITUENTS IN

00:53:33.743 --> 00:53:35.912
SAN FRANCISCO, FOLKS WHO HAVE

00:53:35.979 --> 00:53:38.281
WORKED IN AI, AT STARTUPS, AT

00:53:38.348 --> 00:53:40.650
THE LARGE LABS IN ACADEMIA,

00:53:40.717 --> 00:53:43.019
RESEARCHERS CAME TO ME IN 2023

00:53:43.086 --> 00:53:45.922
AND SAID, WE HAVE AN ISSUE THAT

00:53:45.989 --> 00:53:49.492
THAT AI, THAT GENERATIVE AI IS

00:53:49.559 --> 00:53:52.062
ACCELERATING AND THERE ARE HUGE

00:53:52.128 --> 00:53:54.297
RISKS, AND POLICY MAKERS NEED

00:53:54.364 --> 00:53:56.132
TO DO SOMETHING ABOUT IT. AND

00:53:56.199 --> 00:53:58.568
THAT STARTED OUR WORK THAT

00:53:58.635 --> 00:54:01.972
RESULTED IN SENATE BILL 1047 IN

00:54:02.038 --> 00:54:04.207
2024, LEGISLATION THAT, HAD IT

00:54:04.274 --> 00:54:05.742
NOT BEEN VETOED, WOULD HAVE

00:54:05.809 --> 00:54:07.911
REQUIRED DEVELOPERS OF THE MOST

00:54:07.978 --> 00:54:10.080
POWERFUL AI MODELS TO INVEST IN

00:54:10.146 --> 00:54:12.215
BASIC PRECAUTIONS, INCLUDING

00:54:12.282 --> 00:54:13.683
PRE-DEPLOYMENT SAFETY TESTING,

00:54:13.750 --> 00:54:15.885
RED TEAMING, CYBERSECURITY,

00:54:15.952 --> 00:54:19.022
SAFE CYBERSECURITY SAFEGUARDS,

00:54:19.089 --> 00:54:20.523
SAFEGUARDS TO PREVENT THE

00:54:20.590 --> 00:54:22.292
MISUSE OF DANGEROUS

00:54:22.359 --> 00:54:24.094
CAPABILITIES, POST-DEPLOYMENT

00:54:24.160 --> 00:54:25.562
MONITORING, AND SHUTDOWN

00:54:25.629 --> 00:54:26.696
CAPABILITY REQUIREMENTS. ALSO

00:54:26.763 --> 00:54:28.465
KNOWN AS A KILL SWITCH. THE

00:54:28.531 --> 00:54:31.334
BILL ALSO REQUIRED THAT THE

00:54:31.401 --> 00:54:33.470
THAT THE LABS THE AI LABS

00:54:33.536 --> 00:54:35.238
REPORT THEIR PRECAUTIONARY

00:54:35.305 --> 00:54:36.339
MEASURES TO THE ATTORNEY

00:54:36.406 --> 00:54:37.774
GENERAL WHO COULD TAKE

00:54:37.841 --> 00:54:39.175
ENFORCEMENT ACTION AGAINST THE

00:54:39.242 --> 00:54:41.111
DEVELOPER IF THEIR MODEL WERE

00:54:41.177 --> 00:54:42.779
TO CAUSE SEVERE HARM OR IF

00:54:42.846 --> 00:54:44.014
THEY'RE NEGLIGENT, WERE TO POSE

00:54:44.080 --> 00:54:45.715
AN IMMINENT RISK TO PUBLIC

00:54:45.782 --> 00:54:47.851
SAFETY. THE THE BILL ALSO WOULD

00:54:47.917 --> 00:54:51.121
HAVE ENACTED WHISTLEBLOWER

00:54:51.187 --> 00:54:53.323
PROTECTIONS. AND I WANT TO

00:54:53.390 --> 00:54:54.424
AGAIN THANK THE CHAIR FOR

00:54:54.491 --> 00:54:57.293
WORKING WITH US ON SB 1047,

00:54:57.360 --> 00:54:58.862
WHICH WE SENT TO THE GOVERNOR,

00:54:58.928 --> 00:55:01.531
AND UNFORTUNATELY, IT WAS

00:55:01.598 --> 00:55:03.667
VETOED. THERE WAS HUGE

00:55:03.733 --> 00:55:06.970
OPPOSITION TO SB 1047 FROM THE

00:55:07.037 --> 00:55:09.439
TECH INDUSTRY, JUST A TIDAL

00:55:09.506 --> 00:55:11.675
WAVE OF TECH INDUSTRY

00:55:11.741 --> 00:55:13.476
OPPOSITION. THERE WERE ALSO A

00:55:13.543 --> 00:55:15.645
NUMBER OF MEMBERS OF CONGRESS

00:55:15.712 --> 00:55:20.283
WHO PUBLICLY OPPOSED SB 1047.

00:55:20.350 --> 00:55:21.518
WE WERE CALLED EVERY NAME IN

00:55:21.584 --> 00:55:23.887
THE BOOK. BACK IN 2024, WE WERE

00:55:23.953 --> 00:55:26.089
CALLED DOOMERS. PEOPLE SAID

00:55:26.156 --> 00:55:27.691
THAT WE WERE JUST CRAZY. WE

00:55:27.757 --> 00:55:30.427
WERE CALLED LUDDITES. WE WERE

00:55:30.493 --> 00:55:32.128
ACCUSED OF BEING SCIENCE

00:55:32.195 --> 00:55:34.664
FICTION WRITERS. AND ALTHOUGH

00:55:34.731 --> 00:55:36.666
IT WOULD BE FUN TO BE A SCIENCE

00:55:36.733 --> 00:55:37.667
FICTION WRITER, THAT IS

00:55:37.734 --> 00:55:38.735
CERTAINLY NOT WHAT WE WERE

00:55:38.802 --> 00:55:40.236
DOING. BUT THAT'S THOSE ARE THE

00:55:40.303 --> 00:55:42.072
KINDS OF TERMS THAT WERE THROWN

00:55:42.138 --> 00:55:45.775
AROUND. AND OUR OUR CRITICS

00:55:45.842 --> 00:55:48.011
WOULD ROUTINELY SAY THAT WE

00:55:48.078 --> 00:55:49.145
WERE JUST MAKING IT UP, THAT

00:55:49.212 --> 00:55:51.514
THESE RISKS WERE NOT REAL. WELL,

00:55:51.581 --> 00:55:53.216
THE RISKS ARE REAL. THEY WERE

00:55:53.283 --> 00:55:54.718
REAL THEN. AND THEY ARE EVEN

00:55:54.784 --> 00:55:57.554
MORE REAL NOW. AND WE KNOW THAT

00:55:57.620 --> 00:56:01.224
EVEN BACK THEN, THE TOP EXPERTS

00:56:01.291 --> 00:56:05.128
IN THE WORLD WERE WERE TELLING

00:56:05.195 --> 00:56:07.697
US, WERE TELLING US THAT THESE

00:56:07.764 --> 00:56:09.532
RISKS WERE REAL AND THAT WE

00:56:09.599 --> 00:56:12.035
HAVE TO, AND THAT WE MUST NOT

00:56:12.102 --> 00:56:14.504
IGNORE THEM. EXCUSE ME ONE

00:56:14.571 --> 00:56:18.842
SECOND. AND WE'VE SEEN RECENTLY,

00:56:18.908 --> 00:56:20.977
OF COURSE, WITH THE HUGGING

00:56:21.044 --> 00:56:23.113
FACE INCIDENT AND OTHER

00:56:23.179 --> 00:56:25.715
INCIDENTS, THAT THE RISKS ARE

00:56:25.782 --> 00:56:27.817
NOT JUST REAL, BUT THEY ARE

00:56:27.884 --> 00:56:30.687
STARTING TO MANIFEST WITH AI

00:56:30.754 --> 00:56:34.357
AGENTS THAT ARE ENGAGING IN, IN,

00:56:34.424 --> 00:56:36.493
IN ACTIVITY AND HACKING

00:56:36.559 --> 00:56:37.594
ACTIVITY. THAT WOULD BE A

00:56:37.660 --> 00:56:39.729
FELONY IF COMMITTED BY A HUMAN

00:56:39.796 --> 00:56:42.599
BEING. AS I MENTIONED, 1047 WAS

00:56:42.665 --> 00:56:44.768
ULTIMATELY VETOED, BUT EVEN

00:56:44.834 --> 00:56:46.236
THOUGH IT WAS VETOED AND THAT

00:56:46.302 --> 00:56:48.371
WAS DISAPPOINTING, IT WAS ALSO

00:56:48.438 --> 00:56:50.140
A BIG STEP FORWARD, NOT ONLY

00:56:50.206 --> 00:56:51.941
BECAUSE IT PROVOKED THE

00:56:52.008 --> 00:56:54.544
CONVERSATION AROUND, WHAT DO WE

00:56:54.611 --> 00:56:56.679
MEAN WHEN WE SAY WE WANT PEOPLE

00:56:56.746 --> 00:56:58.515
NOT TO BE HARMED BY TECHNOLOGY,

00:56:58.581 --> 00:57:00.784
INCLUDING AI? BUT THE GOVERNOR,

00:57:00.850 --> 00:57:03.086
IN HIS VETO MESSAGE,

00:57:03.153 --> 00:57:05.321
ACKNOWLEDGED THE RISK AND

00:57:05.388 --> 00:57:07.190
IMPANELED AN EXPERT PANEL TO

00:57:07.257 --> 00:57:09.626
WORK ON RECOMMENDATIONS AROUND

00:57:09.692 --> 00:57:12.495
REGULATING AI FOR SAFETY. THOSE

00:57:12.562 --> 00:57:14.664
RECOMMENDATIONS CAME OUT, AND

00:57:14.731 --> 00:57:16.499
WE WORKED INTENSIVELY WITH THE

00:57:16.566 --> 00:57:20.136
GOVERNOR TO PRODUCE SB 53 AND

00:57:20.203 --> 00:57:21.504
WORKED WITH THIS COMMITTEE AND

00:57:21.571 --> 00:57:23.606
WITH THE CHAIR, AND WE WERE

00:57:23.673 --> 00:57:26.176
ABLE TO GET SB 53 PASSED AND

00:57:26.242 --> 00:57:28.645
SIGNED INTO LAW. AND, OF COURSE,

00:57:28.711 --> 00:57:30.046
THE FIRST IN THE COUNTRY, WHICH

00:57:30.113 --> 00:57:31.281
HAS NOW BEEN REPLICATED IN

00:57:31.347 --> 00:57:35.852
OTHER STATES. AND THAT LAW

00:57:35.919 --> 00:57:38.021
REQUIRES DISCLOSURE OF HOW

00:57:38.087 --> 00:57:39.622
COMPANIES TEST FOR AND TAKE

00:57:39.689 --> 00:57:43.793
INTO ACCOUNT SAFETY, AND ALSO

00:57:43.860 --> 00:57:46.129
TO DISCLOSE CRITICAL INCIDENTS

00:57:46.196 --> 00:57:49.465
TO CAL OES WITHIN A SET PERIOD

00:57:49.532 --> 00:57:53.236
OF TIME. THE SB 53 ALSO

00:57:53.303 --> 00:57:55.738
PROHIBITS FRONTIER DEVELOPERS,

00:57:55.805 --> 00:57:57.540
OR, EXCUSE ME, ENACTS THE

00:57:57.607 --> 00:57:58.308
WHISTLEBLOWER PROTECTIONS THAT

00:57:58.374 --> 00:58:01.477
WE HAD TRIED TO ENACT IN SB

00:58:01.544 --> 00:58:03.546
1047. AND THEN, OF COURSE,

00:58:03.613 --> 00:58:05.682
CREATES CAL COMPUTE PUBLIC

00:58:05.748 --> 00:58:07.917
CLOUD COMPUTING CLUSTER SO WE

00:58:07.984 --> 00:58:10.119
CAN SPREAD INNOVATION AND NOT

00:58:10.186 --> 00:58:11.855
HAVE IT BE SO DOMINATED BY THE

00:58:11.921 --> 00:58:14.490
LARGE PLAYERS. SO IN ADDITION

00:58:14.557 --> 00:58:16.192
TO BEING REPLICATED IN OTHER

00:58:16.259 --> 00:58:17.427
STATES, A CERTAIN INDIVIDUAL

00:58:17.493 --> 00:58:19.796
WHO LIVES IN THE WHITE HOUSE

00:58:19.863 --> 00:58:21.965
WHO BELIEVES THAT THERE SHOULD

00:58:22.031 --> 00:58:25.001
BE NO REGULATION OF AI AND THAT

00:58:25.068 --> 00:58:26.636
IT SHOULD BE ALL

00:58:26.703 --> 00:58:28.071
SELF-REGULATION BY THE INDUSTRY.

00:58:28.137 --> 00:58:30.273
AND I WANT TO BE CLEAR,

00:58:30.340 --> 00:58:32.775
SELF-REGULATION IS NOT A THING.

00:58:32.842 --> 00:58:34.177
SELF-REGULATION RARELY, IF EVER,

00:58:34.244 --> 00:58:35.411
WORKS, AND IT CERTAINLY WILL

00:58:35.478 --> 00:58:37.113
NOT WORK HERE. AND WE MUST

00:58:37.180 --> 00:58:38.882
FIRMLY REJECT SELF-REGULATION

00:58:38.948 --> 00:58:41.351
WHEN IT COMES TO AI. WE AS

00:58:41.417 --> 00:58:43.152
POLICY MAKERS NEED TO ACT ON

00:58:43.219 --> 00:58:45.355
BEHALF OF OUR CONSTITUENTS. AND

00:58:45.421 --> 00:58:47.624
SO PRESIDENT TRUMP HAS TRIED

00:58:47.690 --> 00:58:51.094
FIVE TIMES TO OBLITERATE SB 53

00:58:51.160 --> 00:58:52.562
THROUGH EXECUTIVE ORDER,

00:58:52.629 --> 00:58:54.731
THROUGH AN ATTEMPT AT VARIOUS

00:58:54.797 --> 00:58:56.532
ATTEMPTS TO GET CONGRESS TO

00:58:56.599 --> 00:58:58.334
PASS PREEMPTION LEGISLATION.

00:58:58.401 --> 00:58:59.535
UNFORTUNATELY, WE'VE BEEN ABLE

00:58:59.602 --> 00:59:01.237
TO BEAT THOSE EFFORTS BACK

00:59:01.304 --> 00:59:03.740
EVERY TIME. AS THE SPEAKER SAID,

00:59:03.806 --> 00:59:04.774
FEDERAL LAW SHOULD ESTABLISH

00:59:04.841 --> 00:59:06.075
THE FLOOR, NOT THE CEILING,

00:59:06.142 --> 00:59:07.710
WHEN IT COMES TO PROTECTING THE

00:59:07.777 --> 00:59:11.247
PUBLIC FROM AI. SO I KNOW THAT

00:59:11.314 --> 00:59:12.348
CALIFORNIA WILL CONTINUE TO

00:59:12.415 --> 00:59:15.618
PLAY A PIVOTAL ROLE IN

00:59:15.685 --> 00:59:16.819
ESTABLISHING SAFEGUARDS FOR

00:59:16.886 --> 00:59:18.855
THIS TRANSFORMATIVE TECHNOLOGY

00:59:18.922 --> 00:59:20.523
AND WILL CONTINUE TO RAISE THE

00:59:20.590 --> 00:59:22.692
BAR HIGHER FOR THE NATION AND

00:59:22.759 --> 00:59:24.294
THE WORLD. IN RESPONSIBLE

00:59:24.360 --> 00:59:26.095
TECHNOLOGY POLICY, WE KNOW THAT

00:59:26.162 --> 00:59:28.898
WE NEED ACTION BY CONGRESS. IN

00:59:28.965 --> 00:59:31.200
ADDITION TO ACTION BY

00:59:31.267 --> 00:59:32.568
CALIFORNIA, WE NEED A FEDERAL

00:59:32.635 --> 00:59:33.970
REGULATOR. WE NEED

00:59:34.037 --> 00:59:35.772
INTERNATIONAL AGREEMENTS. THE

00:59:35.838 --> 00:59:38.308
US AND CHINA MUST WORK TOGETHER

00:59:38.374 --> 00:59:39.709
BECAUSE WE HAVE A SHARED

00:59:39.776 --> 00:59:41.077
INTEREST IN MAKING SURE THAT

00:59:41.144 --> 00:59:42.612
CATASTROPHIC HARM DOES NOT

00:59:42.679 --> 00:59:44.047
RESULT FROM ARTIFICIAL

00:59:44.113 --> 00:59:45.114
INTELLIGENCE. I DON'T CARE IF

00:59:45.181 --> 00:59:46.616
YOU ARE AMERICAN OR CHINESE OR

00:59:46.683 --> 00:59:48.384
ANY OTHER NATIONALITY ON THE

00:59:48.451 --> 00:59:49.786
PLANET. IT IS IN ALL OF OUR

00:59:49.852 --> 00:59:51.654
INTERESTS FOR THIS TECHNOLOGY

00:59:51.721 --> 00:59:53.389
TO MAKE LIFE BETTER FOR

00:59:53.456 --> 00:59:55.591
HUMANITY AND NOT TO HARM

00:59:55.658 --> 00:59:59.195
HUMANITY, WE ALSO NEED OUTSIDE

00:59:59.262 --> 01:00:01.097
AUDITING, INDEPENDENT AUDITING

01:00:01.164 --> 01:00:03.833
THAT IS NOT CONTROLLED BY THE

01:00:03.900 --> 01:00:05.668
TECH COMPANIES. I WANT TO

01:00:05.735 --> 01:00:07.070
REALLY ACKNOWLEDGE THE CHAIR

01:00:07.136 --> 01:00:09.305
FOR HER LEADERSHIP. WE INCLUDED

01:00:09.372 --> 01:00:12.175
AUDITING AND SB 1047, BUT THE

01:00:12.241 --> 01:00:13.609
CHAIR HAS REALLY TAKEN IT TO

01:00:13.676 --> 01:00:17.180
THE NEXT LEVEL IN IN ADVOCATING

01:00:17.246 --> 01:00:19.749
FOR OUTSIDE INDEPENDENT AUDITS.

01:00:19.816 --> 01:00:20.883
SO THANK YOU, MADAM CHAIR, AND

01:00:20.950 --> 01:00:22.585
I THINK YOU HAVE BEEN PROVEN

01:00:22.652 --> 01:00:24.020
RIGHT IN THAT, BECAUSE THERE IS

01:00:24.087 --> 01:00:25.922
GROWING SUPPORT FOR THAT NOTION

01:00:25.989 --> 01:00:27.623
THAT YOU WERE REALLY FIGHTING

01:00:27.690 --> 01:00:30.460
TO ACHIEVE FOR SEVERAL YEARS.

01:00:30.526 --> 01:00:33.796
SO I AM OPTIMISTIC THAT THE

01:00:33.863 --> 01:00:34.764
CALIFORNIA LEGISLATURE WILL

01:00:34.831 --> 01:00:37.000
CONTINUE TO LEAD ON TECH

01:00:37.066 --> 01:00:38.768
REGULATION IN GENERAL AND AI

01:00:38.835 --> 01:00:40.370
REGULATION AROUND SAFETY AND

01:00:40.436 --> 01:00:43.172
OTHER ISSUES IN PARTICULAR. AND

01:00:43.239 --> 01:00:45.475
I AM JUST VERY, VERY GRATEFUL

01:00:45.541 --> 01:00:47.810
FOR THIS GROWING COALITION OF

01:00:47.877 --> 01:00:50.713
LEGISLATORS AND ADVOCATES FOR

01:00:50.780 --> 01:00:52.115
MOVING US IN THIS DIRECTION. SO,

01:00:52.181 --> 01:00:53.950
MADAM CHAIR, THANK YOU SO MUCH

01:00:54.017 --> 01:00:55.351
FOR GIVING ME THIS OPPORTUNITY

01:00:55.418 --> 01:00:57.186
TODAY. THANK YOU SO MUCH,

01:00:57.253 --> 01:00:58.955
SENATOR. AND I REALLY YOU'RE

01:00:59.022 --> 01:00:59.856
GETTING APPLAUSE. I DON'T KNOW

01:00:59.922 --> 01:01:01.758
IF YOU CAN HEAR IT, BUT I DO

01:01:01.824 --> 01:01:03.659
WANT TO THANK YOU FOR YOUR

01:01:03.726 --> 01:01:05.828
LEADERSHIP IN THIS SPACE. YOU

01:01:05.895 --> 01:01:06.896
REALLY HAVE BEEN A TIRELESS

01:01:06.963 --> 01:01:08.631
ADVOCATE. I WAS, FOR THOSE WHO

01:01:08.698 --> 01:01:10.533
DON'T KNOW, I WAS THE SENATOR'S

01:01:10.600 --> 01:01:12.635
FLOOR MANAGER FOR HIS BILL ON

01:01:12.702 --> 01:01:14.170
THE ASSEMBLY FLOOR. SO I WORKED

01:01:14.237 --> 01:01:15.838
SIDE BY SIDE WITH HIM TO FIGHT

01:01:15.905 --> 01:01:17.807
TO GET 1047 TO THE GOVERNOR'S

01:01:17.874 --> 01:01:19.475
DESK, WHICH WAS NOT EASY. AND

01:01:19.542 --> 01:01:22.011
HE FOUGHT TIRELESSLY TO MAKE

01:01:22.078 --> 01:01:23.479
THAT HAPPEN. THERE'S NO

01:01:23.546 --> 01:01:25.048
QUESTION IN MY MIND THAT YOU

01:01:25.114 --> 01:01:26.115
HAVE YOU WERE PRESCIENT IN

01:01:26.182 --> 01:01:27.150
UNDERSTANDING HOW QUICKLY THIS

01:01:27.216 --> 01:01:28.451
QUICKLY THIS MIGHT COME TO PASS,

01:01:28.518 --> 01:01:30.219
AND YOU REALLY FOUGHT FOR THE

01:01:30.286 --> 01:01:32.455
SAFETY OF OUR COMMUNITIES. SO I

01:01:32.522 --> 01:01:33.656
APPRECIATE YOU AND APPRECIATE

01:01:33.723 --> 01:01:34.991
THE CONTINUED LEADERSHIP. I

01:01:35.058 --> 01:01:36.559
KNOW YOU WILL SHOW IN THIS

01:01:36.626 --> 01:01:38.895
SPACE, MADAM CHAIR. OF COURSE.

01:01:38.961 --> 01:01:41.130
SO YEAH, WITH THAT, I'LL TURN

01:01:41.197 --> 01:01:45.034
IT OVER TO MEMBERS OF THE

01:01:45.101 --> 01:01:46.469
LEGISLATURE, NOT ALL MEMBERS OF

01:01:46.536 --> 01:01:48.104
THE COMMITTEE. SO I GUESS I'LL

01:01:48.171 --> 01:01:49.739
SAY THAT. ASSEMBLY MEMBER WARD.

01:01:49.806 --> 01:01:50.807
WELL, THANK YOU, MADAM CHAIR.

01:01:50.873 --> 01:01:52.008
MR. SPEAKER, AND SENATOR WIENER,

01:01:52.075 --> 01:01:53.443
FOR THESE OPENING REMARKS. I'LL

01:01:53.509 --> 01:01:54.577
BE BRIEF, BECAUSE I KNOW WE

01:01:54.644 --> 01:01:56.446
HAVE A LOT OF COLLEAGUES HERE

01:01:56.512 --> 01:01:57.513
THAT ALSO MIGHT HAVE SOME

01:01:57.580 --> 01:01:58.581
OPENING THOUGHTS, AND I DON'T

01:01:58.648 --> 01:02:00.450
WANT TO DUPLICATE MANY OF THE

01:02:00.516 --> 01:02:01.617
GOOD WORDS AND AND IMPRESSIONS

01:02:01.684 --> 01:02:02.718
THAT MY COLLEAGUES HAVE ALREADY

01:02:02.785 --> 01:02:04.454
RAISED. BUT OF COURSE, WE ARE

01:02:04.520 --> 01:02:05.588
ON INTERIM RECESS RIGHT NOW

01:02:05.655 --> 01:02:06.989
BACK HOME IN OUR DISTRICTS,

01:02:07.056 --> 01:02:08.291
TALKING WITH OUR CONSTITUENTS,

01:02:08.357 --> 01:02:09.225
HAVING THOSE TOWN HALLS. AND

01:02:09.292 --> 01:02:10.893
THIS IS A REALLY CURRENT ISSUE

01:02:10.960 --> 01:02:12.862
THAT THEY WANT US TO BE ABLE TO

01:02:12.929 --> 01:02:14.163
WORK ON. SO I'M REALLY GRATEFUL

01:02:14.230 --> 01:02:15.798
THAT YOU CALLED THIS HEARING

01:02:15.865 --> 01:02:16.966
RIGHT NOW, BECAUSE TIME DOESN'T

01:02:17.033 --> 01:02:18.034
WAIT, RIGHT? WE NEED TO MAKE

01:02:18.101 --> 01:02:19.802
SURE AS THINGS ARE CONTINUING

01:02:19.869 --> 01:02:20.903
TO EVOLVE, EVEN THOUGH WE ARE

01:02:20.970 --> 01:02:22.071
TECHNICALLY ON RECESS RIGHT NOW,

01:02:22.138 --> 01:02:23.906
THAT WE ARE GETTING A CLEAR

01:02:23.973 --> 01:02:25.007
SNAPSHOT IN TIME FOR WHAT HAS

01:02:25.074 --> 01:02:26.042
HAPPENED, EVEN SINCE WE

01:02:26.109 --> 01:02:27.076
RECESSED ON SEPTEMBER 1ST,

01:02:27.143 --> 01:02:28.778
THINGS HAVE HAPPENED IN THIS

01:02:28.845 --> 01:02:29.979
SPACE. NEWS MEDIA HAS COME OUT,

01:02:30.046 --> 01:02:30.980
INVESTIGATIONS HAVE OCCURRED.

01:02:31.047 --> 01:02:32.081
WE ARE LEARNING MORE AND MORE

01:02:32.148 --> 01:02:34.951
ABOUT WHAT AGENTIC AI, BEYOND

01:02:35.017 --> 01:02:37.053
GENERATIVE AI, IS HAPPENING IN

01:02:37.120 --> 01:02:38.821
OUR COMMUNITIES AND WHAT THAT

01:02:38.888 --> 01:02:40.756
IS MEANING RIGHT NOW FOR SOME

01:02:40.823 --> 01:02:42.058
OF THESE TEST MODELS THAT ARE

01:02:42.125 --> 01:02:43.226
OUT THERE RIGHT NOW. AND ALSO

01:02:43.292 --> 01:02:45.061
WHAT THIS COULD MEAN BY THE

01:02:45.128 --> 01:02:46.229
TIME THAT WE RETURN IN JANUARY

01:02:46.295 --> 01:02:47.964
AND INFORMS THE POLICY WORK

01:02:48.030 --> 01:02:49.732
THAT WE MUST DO, HAVE DEEP

01:02:49.799 --> 01:02:51.434
CONCERN, OF COURSE, FOR WHAT

01:02:51.501 --> 01:02:52.768
WE'RE LEARNING ABOUT, YOU KNOW,

01:02:52.835 --> 01:02:54.303
THE POTENTIAL AUTONOMY AND THE

01:02:54.370 --> 01:02:56.906
RISKS THAT IT PROPOSED POSES TO

01:02:56.973 --> 01:02:59.108
MANY OF OUR COMMUNITIES THAT WE

01:02:59.175 --> 01:03:00.843
SERVE. THE FACT THAT WE CAN

01:03:00.910 --> 01:03:02.645
HAVE THESE ACTIONS REALLY TAKE

01:03:02.712 --> 01:03:04.514
ITEMS OFF OF THE INTERNET, BE

01:03:04.580 --> 01:03:05.581
ABLE TO SCHEDULE MEETINGS, HACK

01:03:05.648 --> 01:03:07.617
INTO THIRD PARTY SITES. THIS

01:03:07.683 --> 01:03:09.552
REALLY MOVES CONVERSATIONS INTO

01:03:09.619 --> 01:03:11.287
EXTREMELY CONCERNING TERRITORY.

01:03:11.354 --> 01:03:13.422
AND WHILE IT RAISES ITS OWN SET

01:03:13.489 --> 01:03:15.291
OF CONCERNS AROUND THE SPREAD

01:03:15.358 --> 01:03:17.093
OF MISINFORMATION AND

01:03:17.160 --> 01:03:18.794
HARASSMENT, AGENTIC AI IS ALSO

01:03:18.861 --> 01:03:20.630
HAVING EXTREMELY CONSEQUENTIAL

01:03:20.696 --> 01:03:21.697
OUTCOMES, ESPECIALLY WHEN THE

01:03:21.764 --> 01:03:23.933
MODEL STARTS TAKING ACTIONS

01:03:24.000 --> 01:03:25.368
OUTSIDE ANY PARAMETERS OR

01:03:25.434 --> 01:03:26.402
INTENTIONS OUTLINED BY ITS

01:03:26.469 --> 01:03:27.503
OPERATOR. AND AGAIN, I THINK

01:03:27.570 --> 01:03:29.639
THIS SPEAKS TO, I THINK YOU USE

01:03:29.705 --> 01:03:31.807
THE WORD PRESCIENT AND FORWARD

01:03:31.874 --> 01:03:33.242
THINKING OBSERVATIONS BY OUR

01:03:33.309 --> 01:03:34.544
COLLEAGUES, SENATOR WIENER, AND

01:03:34.610 --> 01:03:35.878
I WANT TO THANK HIM AGAIN FOR

01:03:35.945 --> 01:03:36.846
HIS LEADERSHIP. SEVERAL

01:03:36.913 --> 01:03:38.714
ATTEMPTS TO BE ABLE TO BEGIN TO

01:03:38.781 --> 01:03:39.815
GET INTO THIS ISSUE, BUT WE

01:03:39.882 --> 01:03:41.717
HAVE TO GET THIS RIGHT. ANY

01:03:41.784 --> 01:03:42.585
CATASTROPHIC HARMS TO

01:03:42.652 --> 01:03:43.719
CALIFORNIANS IN OUR

01:03:43.786 --> 01:03:45.188
CONSTITUENCIES WITHOUT US

01:03:45.254 --> 01:03:46.189
ACTUALLY HAVING DONE OUR DUE

01:03:46.255 --> 01:03:47.490
DILIGENCE AND REALLY TRYING TO

01:03:47.557 --> 01:03:48.724
BE ABLE TO GET AHEAD OF THOSE

01:03:48.791 --> 01:03:50.459
AND PROTECT THOSE WOULD BE

01:03:50.526 --> 01:03:51.561
DERELICT OF OUR DUTIES HERE.

01:03:51.627 --> 01:03:52.728
AND SO I'M GLAD THAT WE'RE

01:03:52.795 --> 01:03:54.564
BEING HERE TODAY. I ALSO WANT

01:03:54.630 --> 01:03:55.698
TO HIGHLIGHT FOR THE NEW YORK

01:03:55.765 --> 01:03:57.567
TIMES, I WANT TO COMMEND THAT

01:03:57.633 --> 01:03:58.734
WE HAVE CLEARLY A MAJORITY OF

01:03:58.801 --> 01:04:00.536
WOMEN ON THIS PANEL THAT ARE

01:04:00.603 --> 01:04:01.571
HELPING TO LEAD THIS

01:04:01.637 --> 01:04:02.471
CONVERSATION, BECAUSE THAT

01:04:02.538 --> 01:04:03.506
MATTERS, MAKING SURE THAT

01:04:03.573 --> 01:04:04.674
EVERYBODY IS HERE TO BE ABLE TO

01:04:04.740 --> 01:04:06.475
SUPPORT OUR KIDS AND OUR

01:04:06.542 --> 01:04:07.510
COMMUNITIES. GRATEFUL FOR ALL

01:04:07.577 --> 01:04:08.611
MY COLLEAGUES THAT HAVE SHOWN

01:04:08.678 --> 01:04:10.546
UP HERE TODAY. I KNOW WE HAVE A

01:04:10.613 --> 01:04:11.781
LOT TO REVIEW. I CERTAINLY HAVE

01:04:11.847 --> 01:04:13.649
A LOT OF QUESTIONS FOR A LOT OF

01:04:13.716 --> 01:04:15.518
OUR PRESENTERS THAT ARE GOING

01:04:15.585 --> 01:04:16.752
TO BE UP HERE, BECAUSE WE HAVE

01:04:16.819 --> 01:04:18.487
TO UNDERSTAND FROM THOSE, FROM

01:04:18.554 --> 01:04:19.488
THOSE EXECUTIVES, FROM THOSE

01:04:19.555 --> 01:04:20.089
REPRESENTATIVES SPECIFICALLY,

01:04:20.156 --> 01:04:22.258
WHERE THINGS ARE HEADED WITH A

01:04:22.325 --> 01:04:23.893
LOT OF THE TECHNOLOGY THAT IS

01:04:23.960 --> 01:04:24.961
OUT THERE RIGHT NOW. AND I

01:04:25.027 --> 01:04:26.896
THINK I HAVE TO UNDERSCORE WHAT

01:04:26.963 --> 01:04:28.097
OUR SPEAKER SAID AS WELL. Tð

01:04:28.164 --> 01:04:29.899
THESE DO NOT HAVE TO BE

01:04:29.966 --> 01:04:30.900
COMPETING PRIORITIES. WHEN WE

01:04:30.967 --> 01:04:32.001
TALK ABOUT THE GROWTH OF THE

01:04:32.068 --> 01:04:33.102
GREAT INDUSTRIES THAT WE HAVE

01:04:33.169 --> 01:04:34.870
HERE AND THE ADVANCEMENTS THAT

01:04:34.937 --> 01:04:36.038
WE HAVE FOR HUMANITY, WITH THE

01:04:36.105 --> 01:04:37.173
INCLUSION OF NEWER AI MODELS,

01:04:37.240 --> 01:04:39.075
WE WANT TO BE ABLE TO SEE THAT

01:04:39.141 --> 01:04:40.443
SUCCEED. WE ALSO ABSOLUTELY

01:04:40.509 --> 01:04:41.877
HAVE TO SHARE IN THE

01:04:41.944 --> 01:04:42.912
RESPONSIBILITIES THAT WE HAVE

01:04:42.979 --> 01:04:44.080
TO MAKE SURE THAT WE'RE NOT

01:04:44.146 --> 01:04:45.181
LETTING GO OF SOMETHING THAT

01:04:45.248 --> 01:04:46.249
AFFECTS OUR PRIVACY, AFFECTS

01:04:46.315 --> 01:04:47.416
OUR CONSTITUTIONAL RIGHTS, OR

01:04:47.483 --> 01:04:49.585
MOST IMPORTANTLY, AFFECTS OUR

01:04:49.652 --> 01:04:51.387
RIGHT AND OUR FREEDOMS TO BE

01:04:51.454 --> 01:04:52.888
WHO WE WANT TO BE ABLE TO GO

01:04:52.955 --> 01:04:54.790
ABOUT OUR COMMUNITY, TO BE ABLE

01:04:54.857 --> 01:04:56.759
TO ACTUALLY SEE A BRIGHT FUTURE

01:04:56.826 --> 01:04:58.961
FOR OURSELVES AND TO KEEP US IN

01:04:59.028 --> 01:05:00.930
EACH OTHER SAFE. AND SO I THINK

01:05:00.997 --> 01:05:02.098
THAT IS THE SHARED COMMITMENT

01:05:02.164 --> 01:05:03.933
THAT WE HAVE HERE AS WE ADDRESS

01:05:04.000 --> 01:05:05.001
SOME OF THESE ISSUES. BUT,

01:05:05.067 --> 01:05:06.102
MADAM, MADAM CHAIR, I WANT TO

01:05:06.168 --> 01:05:07.937
THANK YOU AGAIN FOR CONVENING

01:05:08.004 --> 01:05:09.038
US HERE TODAY. I'M LOOKING

01:05:09.105 --> 01:05:10.172
FORWARD TO THE PRESENTATIONS.

01:05:10.239 --> 01:05:11.140
THANK YOU. AND I APPRECIATED

01:05:11.207 --> 01:05:12.908
THE COMMENT. WHEN THE NEW YORK

01:05:12.975 --> 01:05:14.043
TIMES TOLD ME YESTERDAY WHAT I

01:05:14.110 --> 01:05:16.112
WANT OUT OF AI AS A WOMAN IS A

01:05:16.178 --> 01:05:17.947
HALLOWEEN COSTUME, I WAS PRETTY

01:05:18.014 --> 01:05:19.048
CLEAR THAT'S NOT WHAT I WANT

01:05:19.115 --> 01:05:21.017
OUT OF AI AS A WOMAN. I'M

01:05:21.083 --> 01:05:22.952
PRETTY SURE I WANT IT TO CREATE

01:05:23.019 --> 01:05:24.153
A FUTURE THAT IS WONDERFUL FOR

01:05:24.220 --> 01:05:27.356
MY FAMILY, THAT CREATES ALL THE

01:05:27.423 --> 01:05:28.557
PROMISE AND NOT ALL THE PERIL.

01:05:28.624 --> 01:05:30.626
AND THAT'S WHY WE'RE HERE TODAY

01:05:30.693 --> 01:05:32.428
WITH THAT. ASSEMBLYMEMBER IRWIN.

01:05:32.495 --> 01:05:33.596
WELL, THANK YOU VERY MUCH. AND

01:05:33.663 --> 01:05:36.332
I FIRST WANT TO BEGIN BY

01:05:36.399 --> 01:05:37.833
THANKING THE SPEAKER FOR ALL

01:05:37.900 --> 01:05:40.703
THE WORK ON THE OUTCOMES REVIEW

01:05:40.770 --> 01:05:42.104
PROGRAM. DID A GREAT JOB LAST

01:05:42.171 --> 01:05:44.307
YEAR. AND THIS IS EXACTLY THE

01:05:44.373 --> 01:05:46.242
THE TYPE OF BILL THAT WE SHOULD

01:05:46.309 --> 01:05:48.210
BE LOOKING AT. I WANT TO THANK

01:05:48.277 --> 01:05:50.079
CHAIR BAUER KHN FOR HELPING

01:05:50.146 --> 01:05:51.480
COORDINATE THIS HEARING AND

01:05:51.547 --> 01:05:52.915
SENATOR WIENER FOR HIS

01:05:52.982 --> 01:05:53.983
WILLINGNESS TO PARTICIPATE

01:05:54.050 --> 01:05:55.818
TODAY. GIVEN THE RAPID

01:05:55.885 --> 01:05:56.952
ADVANCEMENTS IN AI, IT'S NO

01:05:57.019 --> 01:05:58.020
SURPRISE THAT WE'RE DISCUSSING

01:05:58.087 --> 01:05:59.755
THE POTENTIAL RAMIFICATIONS OF

01:05:59.822 --> 01:06:01.991
THESE DEVELOPMENTS. JUST OVER

01:06:02.058 --> 01:06:03.759
TWO MONTHS AGO, CHAIR BAUER

01:06:03.826 --> 01:06:05.895
KHN AND I HAD A JOINT

01:06:05.961 --> 01:06:07.163
INFORMATIONAL HEARING BETWEEN

01:06:07.229 --> 01:06:08.164
THE PRIVACY COMMITTEE AND THE

01:06:08.230 --> 01:06:09.532
SELECT COMMITTEE ON

01:06:09.598 --> 01:06:10.966
CYBERSECURITY. SHORTLY AFTER

01:06:11.033 --> 01:06:13.069
THE NEWS OF THE HUGGING FACE

01:06:13.135 --> 01:06:15.237
INCIDENT BROKE, THE AMOUNT OF

01:06:15.304 --> 01:06:16.672
INFORMATION THAT HAS EMERGED

01:06:16.739 --> 01:06:19.942
SINCE THAT INCIDENT HAS BEEN

01:06:20.009 --> 01:06:21.777
ASTOUNDING. ONE OVERARCHING

01:06:21.844 --> 01:06:23.512
THEME HAS BEEN THAT IT IS CLEAR

01:06:23.579 --> 01:06:26.015
THAT WE MUST REVISIT SB 53 IN

01:06:26.082 --> 01:06:27.450
LIGHT OF THESE DEVELOPMENTS.

01:06:27.516 --> 01:06:29.318
THROUGHOUT MY 12 YEARS IN THE

01:06:29.385 --> 01:06:30.353
LEGISLATURE, I'VE BEEN A

01:06:30.419 --> 01:06:32.521
STAUNCH ADVOCATE FOR

01:06:32.588 --> 01:06:34.290
PRIORITIZING CYBERSECURITY, AS

01:06:34.357 --> 01:06:35.491
I'M SURE MY COLLEAGUES HAVE

01:06:35.558 --> 01:06:38.294
HEARD FROM ME MANY TIMES, THESE

01:06:38.361 --> 01:06:39.562
RECENT INCIDENTS, ALONG WITH

01:06:39.628 --> 01:06:40.796
THE DEVELOPMENT ANNOUNCED

01:06:40.863 --> 01:06:42.732
EARLIER THIS YEAR BY ANTHROPIC

01:06:42.798 --> 01:06:44.367
IN RELATION TO MYTHOS, SHOWS

01:06:44.433 --> 01:06:46.235
THAT THE LEGISLATURE MUST

01:06:46.302 --> 01:06:48.704
CONTINUE TO PRIORITIZE

01:06:48.771 --> 01:06:50.606
CYBERSECURITY. AS I DEPART FROM

01:06:50.673 --> 01:06:51.941
THE ASSEMBLY NEXT MONTH, I

01:06:52.007 --> 01:06:53.709
TRUST MY COLLEAGUES WILL

01:06:53.776 --> 01:06:55.911
CONTINUE TO MOVE THIS WORK

01:06:55.978 --> 01:06:57.046
FORWARD. SO AGAIN, I WANT TO

01:06:57.113 --> 01:06:58.381
THANK SENATOR WIENER FOR

01:06:58.447 --> 01:07:00.950
JOINING US, AS WELL AS ALL THE

01:07:01.016 --> 01:07:02.685
PANELISTS THAT ARE WITH US

01:07:02.752 --> 01:07:04.787
TODAY. THANK YOU, ASSEMBLYMAN.

01:07:04.854 --> 01:07:06.589
AND I HAVE MADE A COMMITMENT TO

01:07:06.655 --> 01:07:07.656
ASSEMBLY MEMBER, AND I WILL

01:07:07.723 --> 01:07:09.191
KEEP MY PROMISE THAT I AM

01:07:09.258 --> 01:07:10.693
TAKING THE TORCH FROM HER ON

01:07:10.760 --> 01:07:11.761
CYBERSECURITY, AND WE ARE GOING

01:07:11.827 --> 01:07:13.562
TO TAKE IT VERY SERIOUSLY. BUT

01:07:13.629 --> 01:07:14.830
I THINK IT'S AN IMPORTANT NOTE

01:07:14.897 --> 01:07:16.565
ABOUT THE JOINT HEARING WE HAD

01:07:16.632 --> 01:07:17.633
ON CYBERSECURITY IN AUGUST. IT

01:07:17.700 --> 01:07:18.934
WAS AFTER THE HUGGING FACE

01:07:19.001 --> 01:07:19.869
INCIDENT. THE HUGGING FACE DID

01:07:19.935 --> 01:07:21.704
COME UP IN THE HEARING. I WILL

01:07:21.771 --> 01:07:22.905
TELL YOU, THE INFORMATION WE

01:07:22.972 --> 01:07:24.573
RECEIVED IN THAT HEARING WAS

01:07:24.640 --> 01:07:25.408
SIGNIFICANTLY DIFFERENT THAN

01:07:25.474 --> 01:07:26.842
WHAT WE WILL HEAR FROM METER

01:07:26.909 --> 01:07:29.478
TODAY. AND SO I THINK ONE OF

01:07:29.545 --> 01:07:31.147
THE THINGS THAT HAS BEEN MOST

01:07:31.213 --> 01:07:34.116
CONCERNING TO ME ABOUT WHERE WE

01:07:34.183 --> 01:07:35.918
SIT WITH AI IS TWO THINGS. ONE,

01:07:35.985 --> 01:07:38.788
THAT WE ARE SEEING THESE AGENTS

01:07:38.854 --> 01:07:39.855
ACTING IN WAYS THAT ARE

01:07:39.922 --> 01:07:40.956
MISALIGNED WITH THE VALUES OF

01:07:41.023 --> 01:07:42.358
THIS STATE, FRANKLY, WITH THE

01:07:42.425 --> 01:07:45.261
LAWS OF THIS STATE, AND

01:07:45.327 --> 01:07:46.328
SECONDARILY, THAT IT APPEARS

01:07:46.395 --> 01:07:48.097
THAT WHEN THESE AGENTS ARE IN

01:07:48.164 --> 01:07:49.565
TESTING, THE COMPANIES DON'T

01:07:49.632 --> 01:07:51.367
SEEM TO KNOW WHAT THEY'RE DOING

01:07:51.434 --> 01:07:52.735
AND DON'T SEEM TO KNOW WHAT

01:07:52.802 --> 01:07:53.903
THEY'RE DOING FOR POTENTIALLY

01:07:53.969 --> 01:07:56.405
WEEKS TO MONTHS. AND SO THAT

01:07:56.472 --> 01:07:57.473
RAISES THE QUESTION OF HOW

01:07:57.540 --> 01:07:58.641
EFFECTIVE IS THE TESTING IF WE

01:07:58.707 --> 01:08:00.476
DON'T KNOW WHAT IS HAPPENING IN

01:08:00.543 --> 01:08:01.477
THE TESTING ENVIRONMENT? AND

01:08:01.544 --> 01:08:03.379
THAT'S PART OF WHAT I THINK WE

01:08:03.446 --> 01:08:04.613
NEED TO DIG INTO TODAY. BUT I

01:08:04.680 --> 01:08:06.449
THINK IT IS REALLY IMPORTANT TO

01:08:06.515 --> 01:08:07.583
NOTE THAT WHEN WE HAD THAT

01:08:07.650 --> 01:08:09.485
HEARING IN AUGUST, WE DID NOT

01:08:09.552 --> 01:08:10.553
HAVE A FULL SCOPE OF

01:08:10.619 --> 01:08:11.520
UNDERSTANDING RELATED TO THE

01:08:11.587 --> 01:08:12.688
HUGGING FACE INCIDENT THAT WE

01:08:12.755 --> 01:08:14.457
NOW HAVE, WHICH IS WHY TWO

01:08:14.523 --> 01:08:15.524
ASSEMBLY MEMBER WARDS POINT.

01:08:15.591 --> 01:08:16.459
IT'S IMPORTANT THAT WE

01:08:16.525 --> 01:08:17.426
RECONVENED. ASSEMBLY MEMBER

01:08:17.493 --> 01:08:19.662
PELLERIN. THANK YOU. I WANT TO

01:08:19.728 --> 01:08:20.830
THANK THE CHAIR FOR PUTTING

01:08:20.896 --> 01:08:22.298
THIS HEARING TOGETHER TODAY.

01:08:22.364 --> 01:08:24.166
AND THANK YOU TO THE SPEAKER

01:08:24.233 --> 01:08:25.568
FOR BEING HERE. I'VE ENJOYED

01:08:25.634 --> 01:08:26.402
BEING ON THIS COMMITTEE. I'VE

01:08:26.469 --> 01:08:28.137
LEARNED A LOT IN MY TIME

01:08:28.204 --> 01:08:30.239
SERVING ON PRIVACY, AND MY

01:08:30.306 --> 01:08:31.440
CONSTITUENTS ARE ALSO VERY,

01:08:31.507 --> 01:08:32.741
VERY CONCERNED ABOUT WHAT'S

01:08:32.808 --> 01:08:34.877
HAPPENING IN THIS SPACE. I MET

01:08:34.944 --> 01:08:36.745
WITH A COUPLE OF UCSC

01:08:36.812 --> 01:08:38.481
PROFESSORS WHO RAISED SOME

01:08:38.547 --> 01:08:39.882
CONCERNS THAT I JUST NEVER EVEN

01:08:39.949 --> 01:08:41.417
THOUGHT POSSIBLE IN THIS WORLD,

01:08:41.484 --> 01:08:43.185
AND IT'S JUST FRIGHTENING TO ME.

01:08:43.252 --> 01:08:45.421
AND OBVIOUSLY, OUR AI LANDSCAPE

01:08:45.488 --> 01:08:47.289
HAS CHANGED SIGNIFICANTLY SINCE

01:08:47.356 --> 01:08:49.291
THE SIGNING OF SB 53, AND EVEN

01:08:49.358 --> 01:08:51.961
MORE SO SINCE 1047 WAS VETOED

01:08:52.027 --> 01:08:54.763
IN 2024. AND I JUST WANT TO

01:08:54.830 --> 01:08:56.499
JUST THE BACKGROUND PAPER WAS

01:08:56.565 --> 01:08:57.566
EXCEPTIONAL. THANK YOU SO MUCH

01:08:57.633 --> 01:08:59.768
FOR THAT. CLEARLY OUTLINED THAT

01:08:59.835 --> 01:09:01.203
THERE'S SO MUCH MORE WORK TO BE

01:09:01.270 --> 01:09:03.639
DONE. AND, YOU KNOW, WE NEED TO

01:09:03.706 --> 01:09:05.875
HAVE THOSE CLEAR AND STRICT

01:09:05.941 --> 01:09:06.909
GUIDELINES FOR DISCLOSURE OF

01:09:06.976 --> 01:09:08.811
THESE ROGUE INCIDENTS. AND JUST

01:09:08.878 --> 01:09:10.212
LOOKING AT THE NEW YORK TIMES,

01:09:10.279 --> 01:09:11.614
WHERE ARTIFICIAL INTELLIGENCE

01:09:11.680 --> 01:09:13.816
AGENTS, ACTING ON THEIR OWN TRY

01:09:13.883 --> 01:09:15.718
TO GAIN ACCESS TO A RANGE OF

01:09:15.784 --> 01:09:17.052
GOVERNMENT WEBSITES, FILLED OUT

01:09:17.119 --> 01:09:19.588
VISA APPLICATIONS AND SENT IN A

01:09:19.655 --> 01:09:21.724
FAKE HOMICIDE TIP TO THE POLICE

01:09:21.790 --> 01:09:23.192
DEPARTMENT IN PHILADELPHIA. I

01:09:23.259 --> 01:09:25.027
MEAN, THIS IS INSANE THAT THIS

01:09:25.094 --> 01:09:27.162
IS HAPPENING. AND I HAVE GRAVE

01:09:27.229 --> 01:09:29.231
CONCERN AROUND AI AND ELECTIONS.

01:09:29.298 --> 01:09:30.699
AND I WAS ABLE TO GET SOME

01:09:30.766 --> 01:09:32.501
LEGISLATION THROUGH TO PROTECT

01:09:32.568 --> 01:09:35.070
IN THAT SPACE. BUT REALLY, YOU

01:09:35.137 --> 01:09:37.139
KNOW, WE CANNOT WE NEED TO

01:09:37.206 --> 01:09:38.941
EXPLORE WHAT CONSEQUENCES NEED

01:09:39.008 --> 01:09:40.576
TO BE ATTACHED TO THESE KINDS

01:09:40.643 --> 01:09:42.611
OF ROGUE INCIDENTS. AND

01:09:42.678 --> 01:09:43.779
CERTAINLY, AS SENATOR WIENER

01:09:43.846 --> 01:09:45.481
SAID, IF WE WE WOULDN'T ACCEPT

01:09:45.548 --> 01:09:47.583
THESE ACTIONS FROM A HUMAN. SO

01:09:47.650 --> 01:09:50.152
WHY ARE WE ACCEPTING IT FROM AI?

01:09:50.219 --> 01:09:51.887
SO I LOOK FORWARD TO TODAY'S

01:09:51.954 --> 01:09:53.422
PANEL AND DISCUSSION, AND LOOK

01:09:53.489 --> 01:09:54.790
FORWARD TO LEARNING MORE AND

01:09:54.857 --> 01:09:55.925
FIGURE OUT A PATH FORWARD IN

01:09:55.991 --> 01:09:57.493
THIS. THANK YOU SO MUCH. THANK

01:09:57.560 --> 01:09:58.394
YOU. ASSEMBLY MEMBER

01:09:58.460 --> 01:10:00.229
ASSEMBLYMEMBER PATEL. YES. I

01:10:00.296 --> 01:10:02.398
WANT TO FIRST OPEN BY THANKING

01:10:02.464 --> 01:10:03.899
THE CHAIR FOR INCLUDING ME IN

01:10:03.966 --> 01:10:05.968
THIS HEARING TODAY. I'M NOT

01:10:06.035 --> 01:10:07.436
NORMALLY A TYPICAL MEMBER OF

01:10:07.503 --> 01:10:08.971
THIS COMMITTEE, AND I HAVEN'T

01:10:09.038 --> 01:10:10.205
HAD THE PRIVILEGE OF ENGAGING

01:10:10.272 --> 01:10:11.407
IN THE CONVERSATIONS AND

01:10:11.473 --> 01:10:13.876
DEBATES AROUND SB 53 IN THIS

01:10:13.943 --> 01:10:16.111
HEARING FORMAT. SO REALLY DEEP

01:10:16.178 --> 01:10:17.179
GRATITUDE FOR INCLUDING ME IN

01:10:17.246 --> 01:10:18.981
THIS CONVERSATION. I AM

01:10:19.048 --> 01:10:19.715
DEFINITELY LOOKING FORWARD TO

01:10:19.782 --> 01:10:21.450
HEARING THE WITNESS TESTIMONY

01:10:21.517 --> 01:10:22.618
THAT YOU HAVE LINED UP FOR OUR

01:10:22.685 --> 01:10:24.720
HEARING TODAY. I ALSO

01:10:24.787 --> 01:10:27.623
APPRECIATE THE CHAIR'S APPROACH

01:10:27.690 --> 01:10:29.391
SIMPLY TO THIS ISSUE AND

01:10:29.458 --> 01:10:30.826
APPROACHING IT WITH SUCH

01:10:30.893 --> 01:10:32.628
IMPORTANCE. AND I SEE THAT AND

01:10:32.695 --> 01:10:34.797
HER REPUTATION OF APPROACHING

01:10:34.863 --> 01:10:35.831
POLICYMAKING IN A SCIENTIFIC

01:10:35.898 --> 01:10:39.101
AND METHODICAL WAY IS DEEPLY

01:10:39.168 --> 01:10:40.202
RESONATES WITH MY APPROACH TO

01:10:40.269 --> 01:10:42.671
POLICY MAKING. THE IMPLICATIONS

01:10:42.738 --> 01:10:44.873
AROUND THIS TECHNOLOGY CANNOT

01:10:44.940 --> 01:10:47.042
BE OVERSTATED, AND THE INCIDENT

01:10:47.109 --> 01:10:49.178
OF AT HUGGING FACE OF ONE

01:10:49.244 --> 01:10:50.279
EXAMPLE, BUT ALSO OTHER

01:10:50.346 --> 01:10:51.714
EXAMPLES THAT WE SAW IN THE

01:10:51.780 --> 01:10:53.482
WONDERFULLY CRAFTED BACKGROUND

01:10:53.549 --> 01:10:55.284
PAPER SHOW HOW QUICKLY THE

01:10:55.351 --> 01:10:56.785
PLAUSIBLE DANGERS HAVE BECOME

01:10:56.852 --> 01:10:58.187
REAL. THEY'VE SHIFTED FROM

01:10:58.253 --> 01:11:00.289
HYPOTHETICAL TO REAL. I AM

01:11:00.356 --> 01:11:01.423
GRATEFUL THAT THE CHAIR IS

01:11:01.490 --> 01:11:02.758
HAVING THIS HEARING TODAY, SO

01:11:02.825 --> 01:11:04.360
WE CAN ENSURE THAT CALIFORNIA

01:11:04.426 --> 01:11:05.761
GETS IT RIGHT FOR OUR

01:11:05.828 --> 01:11:07.096
CONSTITUENTS. WE DON'T WANT TO

01:11:07.162 --> 01:11:08.964
CATCH OURSELVES BEING REACTIVE

01:11:09.031 --> 01:11:10.165
AFTER THAT CATASTROPHIC EVENT

01:11:10.232 --> 01:11:11.600
HAS ALREADY HAPPENED. WE WANT

01:11:11.667 --> 01:11:13.235
TO BE THOUGHTFULLY RESPONSIVE,

01:11:13.302 --> 01:11:14.703
AND I APPRECIATE ENGAGING IN

01:11:14.770 --> 01:11:15.871
THIS CONVERSATION TODAY AND

01:11:15.938 --> 01:11:17.940
INCLUDING ME AT THE TABLE. I'M

01:11:18.007 --> 01:11:20.075
PARTICULARLY INTERESTED IN

01:11:20.142 --> 01:11:21.076
OPPORTUNITIES TO IMPROVE OUR

01:11:21.143 --> 01:11:22.678
STATE'S CYBERSECURITY, AS YOU

01:11:22.745 --> 01:11:24.513
MENTIONED, AND LOOK FORWARD TO

01:11:24.580 --> 01:11:26.348
WORKING WITH YOU FOR THE NEXT

01:11:26.415 --> 01:11:29.985
TWO YEARS TO COME. THANK YOU.

01:11:30.052 --> 01:11:31.053
ANYONE ELSE? YEAH. SO MEMBER

01:11:31.120 --> 01:11:34.223
WIX. HI EVERYONE. I'M SO

01:11:34.289 --> 01:11:35.324
EXCITED TO BE BACK IN

01:11:35.391 --> 01:11:37.860
SACRAMENTO. I WANT TO THANK THE

01:11:37.926 --> 01:11:39.595
SPEAKER FOR BEING HERE TODAY

01:11:39.662 --> 01:11:41.030
AND FOR HIS LEADERSHIP IN THIS

01:11:41.096 --> 01:11:42.464
SPACE. AND OBVIOUSLY, OUR

01:11:42.531 --> 01:11:44.299
WONDERFUL CHAIR, WHO HAS BEEN

01:11:44.366 --> 01:11:46.168
DOGGED IN HER PURSUIT OF MAKING

01:11:46.235 --> 01:11:47.536
SURE WE HAVE SAFETY. AND ALSO,

01:11:47.603 --> 01:11:49.738
SENATOR WIENER, FOR TRULY BEING

01:11:49.805 --> 01:11:51.106
ONE OF THE FIRST LEADERS IN

01:11:51.173 --> 01:11:54.043
THIS COUNTRY TO DRIVE FORTH

01:11:54.109 --> 01:11:55.044
COMPREHENSIVE SAFETY

01:11:55.110 --> 01:11:57.279
LEGISLATION. AND HE IS HE IS AN

01:11:57.346 --> 01:11:59.782
OG IN THIS SPACE. AND SO THANK

01:11:59.848 --> 01:12:01.483
YOU, SENATOR WIENER, FOR ALL

01:12:01.550 --> 01:12:03.385
YOUR WORK IN THIS SPACE AND SO

01:12:03.452 --> 01:12:05.154
MANY OTHER AMAZING,

01:12:05.220 --> 01:12:06.255
TRAILBLAZING SPACES THAT YOU

01:12:06.321 --> 01:12:09.491
HAVE LED ON IN THE LEGISLATURE.

01:12:09.558 --> 01:12:11.627
YOU KNOW, I HAVE SAT OPPOSITE

01:12:11.694 --> 01:12:12.861
OF THE BIGGEST TECH COMPANIES

01:12:12.928 --> 01:12:15.931
IN THE WORLD MANY, MANY TIMES

01:12:15.998 --> 01:12:18.033
IN A LOT OF THE WORK THAT WE'VE

01:12:18.100 --> 01:12:19.101
DONE TOGETHER, MADAM CHAIR, AND

01:12:19.168 --> 01:12:21.036
MANY OF YOU ON THE COMMITTEE AS

01:12:21.103 --> 01:12:22.838
WELL, CRAFTING CHILD SAFETY

01:12:22.905 --> 01:12:26.008
LEGISLATION. AND THE MESSAGE TO

01:12:26.075 --> 01:12:27.876
THE COMPANIES IS THIS

01:12:27.943 --> 01:12:30.379
LEGISLATION IS COMING. YOU WILL

01:12:30.446 --> 01:12:32.181
SEE A BODY OF WORK COMING OUT

01:12:32.247 --> 01:12:33.949
OF THE ASSEMBLY, PRESUMABLY THE

01:12:34.016 --> 01:12:35.684
SENATE AS WELL, TO MAKE SURE

01:12:35.751 --> 01:12:37.853
THAT OUR COMMUNITIES ARE SAFE.

01:12:37.920 --> 01:12:39.588
AND I THINK MISS PELLERIN SAID

01:12:39.655 --> 01:12:41.490
IT BEST, LIKE WE WOULDN'T ALLOW,

01:12:41.557 --> 01:12:43.992
YOU KNOW, HUMAN BEINGS TO

01:12:44.059 --> 01:12:45.127
CONDUCT THEMSELVES IN THIS WAY.

01:12:45.194 --> 01:12:46.395
WHY SHOULD WE ALLOW COMPANIES

01:12:46.462 --> 01:12:47.629
TO, ESPECIALLY THE RICHEST

01:12:47.696 --> 01:12:49.398
COMPANIES IN THE WORLD? SO

01:12:49.465 --> 01:12:50.899
LEGISLATION IS COMING. SAFETY

01:12:50.966 --> 01:12:52.334
IS OF THE UTMOST IMPORTANCE TO

01:12:52.401 --> 01:12:53.836
US AS LAWMAKERS. IT IS LIKE THE

01:12:53.902 --> 01:12:55.604
MOST CRITICAL THING WE CAN DO

01:12:55.671 --> 01:12:56.672
IS MAKE SURE THAT OUR

01:12:56.739 --> 01:12:57.740
COMMUNITIES ARE SAFE. AND SO I

01:12:57.806 --> 01:12:59.808
IMPLORE YOU TO COME TO THE

01:12:59.875 --> 01:13:02.411
TABLE AND NOT TELL US IT'S NOT

01:13:02.478 --> 01:13:04.346
TECHNOLOGICALLY FEASIBLE. DON'T

01:13:04.413 --> 01:13:05.647
TELL IT. DON'T DON'T HIDE

01:13:05.714 --> 01:13:08.884
BEHIND THE TRADE ASSOCIATIONS.

01:13:08.951 --> 01:13:10.185
COME TO THE TABLE WILLINGNESS

01:13:10.252 --> 01:13:11.820
TO CRAFT MEANINGFUL LEGISLATION

01:13:11.887 --> 01:13:13.188
THAT'S GOING TO BE EFFECTIVE

01:13:13.255 --> 01:13:14.656
BECAUSE WHAT IT WILL HAPPEN IS

01:13:14.723 --> 01:13:16.558
THIS WILL BE THE MODEL FOR THE

01:13:16.625 --> 01:13:17.726
REST OF THE COUNTRY AND FOR

01:13:17.793 --> 01:13:19.294
MANY STATES AND FRANKLY,

01:13:19.361 --> 01:13:21.130
INTERNATIONALLY. AND WE ARE

01:13:21.196 --> 01:13:22.364
TALKING TO OUR GLOBAL PARTNERS

01:13:22.431 --> 01:13:24.333
ALL THE TIME. SO IT'S COMING.

01:13:24.399 --> 01:13:25.534
WE'RE REASONABLE HUMANS. WE'RE

01:13:25.601 --> 01:13:28.003
NOT TRYING TO REGULATE YOU OUT

01:13:28.070 --> 01:13:30.405
OF EXISTENCE. WE ARE PROUD OF

01:13:30.472 --> 01:13:31.440
THE TECH INDUSTRY HERE IN

01:13:31.507 --> 01:13:32.474
CALIFORNIA. IT'S A HUGE

01:13:32.541 --> 01:13:34.376
ECONOMIC BASE FOR US. A LOT OF

01:13:34.443 --> 01:13:35.477
THE TOOLS THAT YOU ARE

01:13:35.544 --> 01:13:37.045
DEVELOPING AND CREATING HAVE

01:13:37.112 --> 01:13:38.380
THE ABILITY TO CHANGE THE WORLD.

01:13:38.447 --> 01:13:40.149
AND I SEE THE PROMISE IN ALL OF

01:13:40.215 --> 01:13:42.117
THAT, AND WE HAVE TO KEEP OUR

01:13:42.184 --> 01:13:43.185
COMMUNITY SAFE. AND I BELIEVE

01:13:43.252 --> 01:13:45.187
WE CAN DO BOTH OF THOSE THINGS.

01:13:45.254 --> 01:13:47.089
AND SO I THINK WE WILL APPROACH

01:13:47.156 --> 01:13:48.490
IT WITH THOUGHTFULNESS. AND WE

01:13:48.557 --> 01:13:49.958
AND WE'VE WORKED WITH MANY OF

01:13:50.025 --> 01:13:51.727
THESE TECH COMPANIES. WE HOPE

01:13:51.794 --> 01:13:53.362
THAT THEY WILL COME TO THE

01:13:53.428 --> 01:13:54.963
TABLE. MANY OF THEM HAVE IN

01:13:55.030 --> 01:13:56.765
PRIOR LEGISLATION AS WELL. BUT

01:13:56.832 --> 01:13:58.767
I ANTICIPATE NEXT YEAR WILL BE

01:13:58.834 --> 01:13:59.935
A BIG YEAR IN TERMS OF

01:14:00.002 --> 01:14:00.936
LEGISLATION, BECAUSE, FRANKLY,

01:14:01.003 --> 01:14:01.937
OUR CONSTITUENTS ARE DEMANDING

01:14:02.004 --> 01:14:03.272
IT, AND THAT ULTIMATELY IS WHO

01:14:03.338 --> 01:14:05.073
WE SERVE UP HERE AS OUR

01:14:05.140 --> 01:14:06.108
CONSTITUENTS. THEY ARE OUR

01:14:06.175 --> 01:14:07.342
BOSSES, AND WE LOOK FORWARD TO

01:14:07.409 --> 01:14:09.344
DOING OUR CONSTITUENTS BIDDING

01:14:09.411 --> 01:14:11.113
NEXT YEAR LEGISLATIVELY. THANK

01:14:11.180 --> 01:14:12.014
YOU, ASSEMBLYMEMBER, AND THANK

01:14:12.080 --> 01:14:13.148
YOU FOR YOUR LEADERSHIP,

01:14:13.215 --> 01:14:14.116
ESPECIALLY IN THE CHILDREN'S

01:14:14.183 --> 01:14:16.118
SAFETY REALM. AND I WANT TO

01:14:16.185 --> 01:14:17.452
THANK IN ADVANCE THE COMPANIES

01:14:17.519 --> 01:14:19.454
FOR BEING HERE. THE LARGE MANY

01:14:19.521 --> 01:14:20.556
LARGE COMPANIES ARE HERE TODAY

01:14:20.622 --> 01:14:22.491
TO HELP SHED LIGHT ON THE ISSUE

01:14:22.558 --> 01:14:24.760
BEFORE US. SO ALWAYS APPRECIATE

01:14:24.827 --> 01:14:25.994
WHEN THEY SHOW UP TO HAVE THIS

01:14:26.061 --> 01:14:27.129
CONVERSATION WITH US. AND WITH

01:14:27.196 --> 01:14:28.864
THAT. ASSEMBLYMEMBER BONTA.

01:14:28.931 --> 01:14:30.065
THANK YOU SO MUCH, CHAIR. AND I

01:14:30.132 --> 01:14:31.934
ALSO WANT TO JUST THANK THE

01:14:32.000 --> 01:14:32.968
SPEAKER FOR CREATING A SPACE

01:14:33.035 --> 01:14:35.003
FOR US TO BE ABLE TO HAVE THIS

01:14:35.070 --> 01:14:36.305
OUTCOMES REVIEW OPPORTUNITY TO

01:14:36.371 --> 01:14:37.773
REVIEW THIS LEGISLATION AND, OF

01:14:37.840 --> 01:14:40.008
COURSE, TO SENATOR WIENER FOR

01:14:40.075 --> 01:14:42.511
BEING SO FORWARD THINKING AND

01:14:42.578 --> 01:14:43.912
PRESENTING SOMETHING TO US THAT

01:14:43.979 --> 01:14:45.347
WE WOULD HAVE AN OPPORTUNITY TO

01:14:45.414 --> 01:14:47.482
REACT TO. AND, OF COURSE, OUR

01:14:47.549 --> 01:14:49.017
CHAIR, I'VE APPRECIATED THE

01:14:49.084 --> 01:14:50.352
FACT THAT WE'VE HAD AN

01:14:50.419 --> 01:14:52.187
OPPORTUNITY TO HOLD A JOINT

01:14:52.254 --> 01:14:53.989
HEARING AROUND HEALTH CARE,

01:14:54.056 --> 01:14:56.658
PRIVACY AND AI, AND WHICH LED

01:14:56.725 --> 01:14:58.727
TO OUR BEING ABLE TO PASS TWO

01:14:58.794 --> 01:15:00.562
PIECES OF LEGISLATION, AB 489,

01:15:00.629 --> 01:15:02.965
WHICH PROHIBITED AI SYSTEMS OR

01:15:03.031 --> 01:15:04.833
SIMILAR TECHNOLOGIES FROM

01:15:04.900 --> 01:15:05.534
REPRESENTING THEMSELVES AS

01:15:05.601 --> 01:15:08.036
HEALTH PROFESSIONALS, AND AB

01:15:08.103 --> 01:15:09.504
1979, REQUIRING HEALTH CARE

01:15:09.571 --> 01:15:11.240
CHATBOTS TO COMPLY WITH PRIVACY

01:15:11.306 --> 01:15:12.908
LAWS AND PRESERVE LICENSED

01:15:12.975 --> 01:15:14.176
HEALTH PROFESSIONALS.

01:15:14.243 --> 01:15:15.277
INDEPENDENT JUDGMENT, WHICH I

01:15:15.344 --> 01:15:18.447
THINK WERE CRITICAL AND WERE

01:15:18.513 --> 01:15:20.716
1979 RECENTLY SIGNED INTO LAW

01:15:20.782 --> 01:15:23.785
BY OUR GOVERNOR. WE ARE AT A

01:15:23.852 --> 01:15:27.122
MOMENT WHERE, ONCE AGAIN, IN MY

01:15:27.189 --> 01:15:27.789
ESTIMATION, THE FEDERAL

01:15:27.856 --> 01:15:29.291
GOVERNMENT HAS DECIDED TO

01:15:29.358 --> 01:15:32.160
ABDICATE ITS RESPONSIBILITY TO

01:15:32.227 --> 01:15:34.663
AMERICANS, TO OUR GLOBAL

01:15:34.730 --> 01:15:36.465
CITIZENS, AND CERTAINLY TO

01:15:36.531 --> 01:15:38.667
CALIFORNIANS. AND SO IT'S NOT

01:15:38.734 --> 01:15:40.068
SURPRISING TO ME THAT YET AGAIN,

01:15:40.135 --> 01:15:41.503
CALIFORNIA FINDS ITSELF IN A

01:15:41.570 --> 01:15:44.039
PLACE WHERE WE CAN OFFER BOTH

01:15:44.106 --> 01:15:46.208
INSIGHTS AND A PATHWAY TO

01:15:46.275 --> 01:15:48.010
ENSURE THAT WE HAVE THE KINDS

01:15:48.076 --> 01:15:50.512
OF GUARDRAILS AND FRAMEWORKS IN

01:15:50.579 --> 01:15:53.081
PLACE TO BE ABLE TO SUPPORT OUR

01:15:53.148 --> 01:15:54.383
CONSUMERS, MAKE SURE THAT OUR

01:15:54.449 --> 01:15:55.884
CONSUMERS HAVE THE KIND OF

01:15:55.951 --> 01:15:57.019
PROTECTIONS THAT THEY SO

01:15:57.085 --> 01:15:59.922
DESERVE. I'VE BEEN VERY HAPPY

01:15:59.988 --> 01:16:01.356
TO JOIN ASSEMBLYMEMBER WICKS

01:16:01.423 --> 01:16:03.091
AND BAUER-KAHAN, ESPECIALLY

01:16:03.158 --> 01:16:05.260
AROUND MANY, AND ASSEMBLY

01:16:05.327 --> 01:16:06.795
MEMBER LOWENTHAL, AROUND MANY

01:16:06.862 --> 01:16:08.563
OF THE PIECES OF LEGISLATION TO

01:16:08.630 --> 01:16:10.666
ENSURE THAT OUR CHILDREN, IN

01:16:10.732 --> 01:16:12.100
PARTICULAR, ARE SAFE IN THIS

01:16:12.167 --> 01:16:14.069
SPACE. AND AND WE WILL CONTINUE

01:16:14.136 --> 01:16:16.071
TO DO THAT WORK. FOR ME, SOME

01:16:16.138 --> 01:16:17.205
OF THE OPEN QUESTIONS THAT I

01:16:17.272 --> 01:16:19.308
THINK STILL REMAIN UNANSWERED,

01:16:19.374 --> 01:16:20.542
AND THAT I KNOW THAT WE WILL

01:16:20.609 --> 01:16:22.945
HAVE AN OPPORTUNITY TO EXPLORE,

01:16:23.011 --> 01:16:24.680
IS NOT ONLY WHAT KIND OF

01:16:24.746 --> 01:16:26.915
REGULATORY FRAMEWORK WE WILL BE

01:16:26.982 --> 01:16:28.283
DEVELOPING HERE IN THE STATE OF

01:16:28.350 --> 01:16:29.685
CALIFORNIA, TO BE ABLE TO LEAD

01:16:29.751 --> 01:16:31.887
THE NATION WITH A PACKAGE OF

01:16:31.954 --> 01:16:32.921
LEGISLATION THAT WILL CERTAINLY

01:16:32.988 --> 01:16:34.423
COME FORWARD AND HAS WILL

01:16:34.489 --> 01:16:36.058
CONTINUE TO BE THE WORK THAT WE

01:16:36.124 --> 01:16:38.493
DO. BUT IT'S ALSO TRYING TO

01:16:38.560 --> 01:16:39.861
UNDERSTAND WHAT KIND OF

01:16:39.928 --> 01:16:40.929
INFRASTRUCTURE AND CAPACITY WE

01:16:40.996 --> 01:16:43.065
NEED TO BUILD INSIDE OUR

01:16:43.131 --> 01:16:45.133
AGENCIES TO ENSURE THAT WE KIND

01:16:45.200 --> 01:16:48.103
OF, THAT WE HAVE THE ABILITY TO

01:16:48.170 --> 01:16:50.272
PROVIDE THE PARTNERED

01:16:50.339 --> 01:16:51.773
REGULATION. I THINK THAT WILL

01:16:51.840 --> 01:16:53.275
NEED TO BE REQUIRED IN ORDER

01:16:53.342 --> 01:16:55.344
FOR US TO BE TO SUCCESSFULLY

01:16:55.410 --> 01:16:58.213
KNOW THAT OUR CONSTITUENTS ARE

01:16:58.280 --> 01:17:00.682
SAFE. ULTIMATELY, ONE OF THE

01:17:00.749 --> 01:17:03.218
CHALLENGES THAT I THINK WE KNOW

01:17:03.285 --> 01:17:05.120
EXISTS, AND THAT WE CAN'T DENY,

01:17:05.187 --> 01:17:08.223
IS THAT THIS TECHNOLOGY IS

01:17:08.290 --> 01:17:09.358
DEVELOPING AT SUCH AN

01:17:09.424 --> 01:17:11.860
EXPONENTIAL PACE THAT IT

01:17:11.927 --> 01:17:14.363
OUTPACES OUR ABILITY AS A

01:17:14.429 --> 01:17:15.831
GOVERNMENT ENTITIES TO BE ABLE

01:17:15.897 --> 01:17:17.332
TO MATCH THAT. AND SO WE HAVE

01:17:17.399 --> 01:17:19.801
TO BE LIGHTNING SPEED, READY TO

01:17:19.868 --> 01:17:22.604
BE ABLE TO PIVOT AND TO MOVE

01:17:22.671 --> 01:17:23.672
INTO THIS SPACE. AND I'M

01:17:23.739 --> 01:17:24.906
THANKFUL THAT WE'VE TAKEN OUR

01:17:24.973 --> 01:17:26.041
TIME DURING THE INTERIM SESSION

01:17:26.108 --> 01:17:28.210
TO COME TOGETHER TO EXPLORE

01:17:28.276 --> 01:17:30.112
SOME OF THE WAYS THAT WE CAN DO

01:17:30.178 --> 01:17:33.115
THAT, TO ENSURE THAT WE FEEL

01:17:33.181 --> 01:17:34.883
SAFER, AND ALSO THAT WE TAKE

01:17:34.950 --> 01:17:36.752
ADVANTAGE OF THE INCREDIBLE

01:17:36.818 --> 01:17:39.187
POWER OF THIS PARTICULAR

01:17:39.254 --> 01:17:42.157
TECHNOLOGY. WITH THE REGULATORY

01:17:42.224 --> 01:17:43.525
FRAMEWORKS. ALSO COMES THE

01:17:43.592 --> 01:17:44.659
ISSUES RELATED TO

01:17:44.726 --> 01:17:46.428
INFRASTRUCTURE DEVELOPMENT. MY

01:17:46.495 --> 01:17:48.296
COMMUNITY IN PARTICULAR IS VERY

01:17:48.363 --> 01:17:49.998
CONCERNED ABOUT THE

01:17:50.065 --> 01:17:51.833
REQUIREMENTS THAT DATA CENTERS

01:17:51.900 --> 01:17:54.669
IN PARTICULAR WILL HAVE THAT

01:17:54.736 --> 01:17:57.906
POWER. THESE AI SYSTEMS. AND

01:17:57.973 --> 01:17:59.808
AND THAT IS ALSO A PIECE OF THE

01:17:59.875 --> 01:18:01.309
CONVERSATION THAT I KNOW THAT

01:18:01.376 --> 01:18:03.678
WE WILL BE WAITING INTO AS A

01:18:03.745 --> 01:18:05.213
LEGISLATURE. AND CERTAINLY I,

01:18:05.280 --> 01:18:07.315
IN MY CAPACITY AS A

01:18:07.382 --> 01:18:09.951
REPRESENTATIVE FOR OAKLAND AND

01:18:10.018 --> 01:18:11.887
THE BAY AREA WILL DO AS WELL.

01:18:11.953 --> 01:18:12.921
SO I REALLY APPRECIATE THE

01:18:12.988 --> 01:18:14.890
SPACE TO BE ABLE TO DO THIS

01:18:14.956 --> 01:18:16.058
WORK. THANK YOU. AND THANK YOU.

01:18:16.124 --> 01:18:18.060
I MEAN THAT IN MANY AREAS, BUT

01:18:18.126 --> 01:18:18.994
ESPECIALLY, I THINK, IN THE

01:18:19.061 --> 01:18:21.029
HEALTH SPACE, YOU HAVE LED IN

01:18:21.096 --> 01:18:22.764
CRITICAL WAYS TO MAKE SURE THAT

01:18:22.831 --> 01:18:24.933
THIS TECHNOLOGY BOTH CAN

01:18:25.000 --> 01:18:26.501
SUPPORT PATIENTS, BUT FRANKLY,

01:18:26.568 --> 01:18:27.536
THAT THEY'RE ALSO PROTECTED

01:18:27.602 --> 01:18:29.237
FROM WHAT IS CURRENTLY AN

01:18:29.304 --> 01:18:32.140
UNREGULATED, AS I SEE IT, FIELD

01:18:32.207 --> 01:18:35.744
OF TECHNOLOGY THAT IS

01:18:35.811 --> 01:18:36.511
SIGNIFICANTLY IMPACTING THE

01:18:36.578 --> 01:18:37.746
PROVISION OF HEALTH CARE. AND

01:18:37.813 --> 01:18:40.015
YOU'VE REALLY DUG IN AND MADE A

01:18:40.082 --> 01:18:41.116
BIG DIFFERENCE. SO I WANT TO

01:18:41.183 --> 01:18:44.419
THANK YOU FOR THAT.

01:18:44.486 --> 01:18:45.454
ASSEMBLYMEMBER. ERRANDS? YES.

01:18:45.520 --> 01:18:47.622
OKAY. THANK YOU, MADAM CHAIR,

01:18:47.689 --> 01:18:49.825
AND MR. SPEAKER, AND TO MY

01:18:49.891 --> 01:18:51.593
COLLEAGUES, I JUST WANTED TO

01:18:51.660 --> 01:18:54.096
QUICKLY ADD, AS THE MEMBER

01:18:54.162 --> 01:18:55.263
REPRESENTING SILICON VALLEY,

01:18:55.330 --> 01:18:59.134
I'M ACUTELY AWARE OF THE

01:18:59.201 --> 01:19:01.670
MAGNITUDE OF THIS PROBLEM, AND

01:19:01.736 --> 01:19:02.838
I'M NOT A STANDING MEMBER OF

01:19:02.904 --> 01:19:04.573
THE PRIVACY COMMITTEE, BUT I

01:19:04.639 --> 01:19:06.041
WANTED TO THANK THE CHAIR FOR

01:19:06.108 --> 01:19:07.809
INVITING ME AND LOOKING FORWARD

01:19:07.876 --> 01:19:09.411
TO THIS DISCUSSION. I JUST WANT

01:19:09.478 --> 01:19:10.712
TO ECHO THE SENTIMENT RAISED BY

01:19:10.779 --> 01:19:12.114
THE CHAIR AND MY COLLEAGUES

01:19:12.180 --> 01:19:14.015
ABOUT HOW UNITED WE ARE,

01:19:14.082 --> 01:19:16.785
WHETHER WE REPRESENT SUNNY SAN

01:19:16.852 --> 01:19:18.954
DIEGO TO SILICON VALLEY, TO THE

01:19:19.020 --> 01:19:20.689
CENTRAL VALLEY, TO THE SAN

01:19:20.755 --> 01:19:23.625
GABRIEL VALLEY, WE HAVE A

01:19:23.692 --> 01:19:24.659
RESPONSIBILITY TO GET THIS

01:19:24.726 --> 01:19:27.262
RIGHT. THIS TECHNOLOGY IS SO

01:19:27.329 --> 01:19:31.199
FAST MOVING, AND I WANT TO

01:19:31.266 --> 01:19:32.234
STAND SHOULDER TO SHOULDER WITH

01:19:32.300 --> 01:19:34.436
MY COLLEAGUES. APPRECIATE THE

01:19:34.503 --> 01:19:36.071
LEADERSHIP OF YOU CONVENING

01:19:36.138 --> 01:19:37.939
THIS IMPORTANT COMMITTEE AND

01:19:38.006 --> 01:19:40.942
WANT TO THANK THE MEMBER FROM

01:19:41.009 --> 01:19:42.711
SOUTHERN CALIFORNIA, MISS IRWIN,

01:19:42.777 --> 01:19:44.813
ALSO, FOR HER WORK AND

01:19:44.880 --> 01:19:45.714
CO-CHAIRING THE TECH CAUCUS

01:19:45.780 --> 01:19:47.382
WITH ME THIS PAST YEAR AND

01:19:47.449 --> 01:19:49.518
CREATING EDUCATION ON JUST HOW

01:19:49.584 --> 01:19:51.086
QUICKLY THIS IS MOVING AND HOW

01:19:51.153 --> 01:19:52.687
WE CAN GET MORE OF OUR

01:19:52.754 --> 01:19:53.855
COLLEAGUES IN THE LEGISLATURE

01:19:53.922 --> 01:19:56.024
TO BE AWARE AND INVOLVED AS

01:19:56.091 --> 01:19:57.459
EARLY AS POSSIBLE IN THIS NEXT

01:19:57.526 --> 01:19:59.594
YEAR, SO WE CAN GET THIS RIGHT

01:19:59.661 --> 01:20:01.062
ON BEHALF OF OUR CONSTITUENTS.

01:20:01.129 --> 01:20:02.497
THANK YOU. THANK YOU. ASSEMBLY

01:20:02.564 --> 01:20:05.367
MEMBER, ASSEMBLY MEMBER SCHULTZ.

01:20:05.433 --> 01:20:06.768
THANK YOU VERY MUCH, MADAM

01:20:06.835 --> 01:20:09.371
CHAIR. THANK YOU, MR. SPEAKER.

01:20:09.437 --> 01:20:11.506
GLAD TO BE HERE. I'LL TRY TO BE

01:20:11.573 --> 01:20:13.041
INCREDIBLY BRIEF. I JUST WANT

01:20:13.108 --> 01:20:14.809
TO THANK SENATOR WIENER FOR

01:20:14.876 --> 01:20:16.912
AUTHORING SB 53, FOR AUTHORING

01:20:16.978 --> 01:20:19.047
SB 1047, WHICH I BELIEVE SHOULD

01:20:19.114 --> 01:20:20.182
HAVE BEEN SIGNED INTO LAW. AND

01:20:20.248 --> 01:20:22.150
TO ALL OF YOU WHO HAVE PASSED

01:20:22.217 --> 01:20:23.151
AND CARRIED VERY SIMILAR

01:20:23.218 --> 01:20:24.186
LEGISLATION IN THE LAST FEW

01:20:24.252 --> 01:20:26.988
YEARS, MY BELIEF IS OUR CURRENT

01:20:27.055 --> 01:20:28.023
INFRASTRUCTURE IS INADEQUATE

01:20:28.089 --> 01:20:29.491
AND ECHO THE COMMENTS OF

01:20:29.558 --> 01:20:31.293
ASSEMBLY MEMBER WICKS IN

01:20:31.359 --> 01:20:32.427
PREPARING THE INDUSTRY THAT

01:20:32.494 --> 01:20:34.162
THERE WILL BE MORE LEGISLATION

01:20:34.229 --> 01:20:35.630
COMING. BUT I WANTED TO REALLY

01:20:35.697 --> 01:20:36.464
CLOSE WITH SAYING ONE THING

01:20:36.531 --> 01:20:38.300
THAT I DON'T KNOW WILL COME UP

01:20:38.366 --> 01:20:40.168
IN THE HEART OF THE DISCUSSION

01:20:40.235 --> 01:20:41.469
TODAY, BUT I DO THINK WE SHOULD

01:20:41.536 --> 01:20:43.171
BE THINKING ABOUT IS THE

01:20:43.238 --> 01:20:44.206
GUARDRAILS OF CIVIL AND

01:20:44.272 --> 01:20:44.806
APPLICABLE CRIMINAL LIABILITY

01:20:44.873 --> 01:20:46.841
IN ALL OF THIS, REPLETE

01:20:46.908 --> 01:20:47.909
THROUGHOUT THE BACKGROUNDER

01:20:47.976 --> 01:20:48.743
TODAY. AND AS I'M SURE WE'RE

01:20:48.810 --> 01:20:50.512
GOING TO DISCUSS, THERE ARE

01:20:50.579 --> 01:20:52.180
ACTIONS THAT ARE BEING TAKEN BY

01:20:52.247 --> 01:20:54.716
THESE AI AGENTS THAT, AS WE'VE

01:20:54.783 --> 01:20:55.917
SAID MANY TIMES, WE WOULDN'T

01:20:55.984 --> 01:20:57.118
TOLERATE IF A HUMAN BEING DID

01:20:57.185 --> 01:21:00.422
IT. THE ISSUE IS, WHAT IF AI

01:21:00.488 --> 01:21:02.958
AGENTS ARE DEVELOPED AND ACT

01:21:03.024 --> 01:21:04.693
AUTONOMOUSLY OR UNEXPECTEDLY?

01:21:04.759 --> 01:21:06.294
NOW, ONE COULD CERTAINLY ARGUE

01:21:06.361 --> 01:21:07.629
THAT IF THE APPLICABLE CRIMINAL

01:21:07.696 --> 01:21:08.697
STANDARD FOR THE CRIME WE'RE

01:21:08.763 --> 01:21:10.131
TALKING ABOUT IS NEGLIGENCE OR

01:21:10.198 --> 01:21:11.967
RECKLESSNESS, PERHAPS COURTS

01:21:12.033 --> 01:21:13.835
COULD EXTEND EXISTING DOCTRINES

01:21:13.902 --> 01:21:15.904
AND SAY THAT LIABILITY ATTACHES.

01:21:15.971 --> 01:21:17.339
BUT WHAT ABOUT CRIMES WITH A

01:21:17.405 --> 01:21:19.574
HEIGHTENED OR SPECIFIC INTENT?

01:21:19.641 --> 01:21:20.709
INTENT TO COMMIT A CRIME?

01:21:20.775 --> 01:21:22.410
KNOWING DISREGARD? THE POINT

01:21:22.477 --> 01:21:23.945
THAT I'M I'M MAKING IS THAT I

01:21:24.012 --> 01:21:25.680
THINK ONE THING THE LEGISLATURE

01:21:25.747 --> 01:21:26.881
WILL NEED TO CONSIDER IN THE

01:21:26.948 --> 01:21:28.850
YEAR AHEAD IS WHETHER WE SHOULD

01:21:28.917 --> 01:21:30.385
LEAVE IT TO THE COURTS TO APPLY

01:21:30.452 --> 01:21:32.287
EXISTING DOCTRINES, OR WHETHER

01:21:32.354 --> 01:21:33.989
IT IS THE IMPERATIVE OF OUR

01:21:34.055 --> 01:21:35.323
LEGISLATIVE BODY TO SAY THAT WE,

01:21:35.390 --> 01:21:37.259
AS A MATTER OF POLICY, NEED TO

01:21:37.325 --> 01:21:38.426
MAKE IT QUITE CLEAR THAT YOU

01:21:38.493 --> 01:21:40.295
WILL BE HELD ACCOUNTABLE IF YOU

01:21:40.362 --> 01:21:41.263
DESIGN SOMETHING THAT IS

01:21:41.329 --> 01:21:42.264
INHERENTLY DANGEROUS TO HUMANS

01:21:42.330 --> 01:21:43.465
HEALTH AND SAFETY. SO THANK YOU,

01:21:43.531 --> 01:21:45.267
MADAM CHAIR. I LOOK FORWARD TO

01:21:45.333 --> 01:21:46.534
BEING PART OF THE CONVERSATION.

01:21:46.601 --> 01:21:47.435
THANK YOU, AS ALWAYS,

01:21:47.502 --> 01:21:48.703
APPRECIATE YOUR DEEP LEGAL

01:21:48.770 --> 01:21:51.539
INSIGHT. AS A LAW NERD MYSELF.

01:21:51.606 --> 01:21:54.843
ASSEMBLYMEMBER'S. ZUBIATE. SO

01:21:54.909 --> 01:21:56.044
FIRST OF ALL, MADAM CHAIR, I

01:21:56.111 --> 01:21:57.946
WANT TO THANK YOU FOR INVITING

01:21:58.013 --> 01:21:59.814
ME TO JOIN YOU. I'M NOT A

01:21:59.881 --> 01:22:00.949
SITTING MEMBER OF THE PRIVACY

01:22:01.016 --> 01:22:02.751
COMMITTEE, AND ONE OF THE

01:22:02.817 --> 01:22:04.853
REASONS WHY I'VE HAD

01:22:04.919 --> 01:22:07.489
CONVERSATIONS WITH THE CHAIR IS

01:22:07.555 --> 01:22:09.224
BECAUSE OF THE SIGNIFICANT

01:22:09.291 --> 01:22:11.760
AMOUNT OF QUESTIONS AND FEAR

01:22:11.826 --> 01:22:13.528
THAT'S COMING FROM WITHIN THE

01:22:13.595 --> 01:22:15.563
COMMUNITY. YOU KNOW, AFTER THE

01:22:15.630 --> 01:22:16.598
HUGGING FACE INCIDENT AND

01:22:16.665 --> 01:22:19.434
BEFORE THAT, MANY, MANY CALLS

01:22:19.501 --> 01:22:21.603
IN THE COMMUNITY. AND SO I WANT

01:22:21.670 --> 01:22:24.072
TO THANK THE SPEAKER FOR, YOU

01:22:24.139 --> 01:22:26.141
KNOW, SUPPORTING THE CHAIR'S

01:22:26.207 --> 01:22:27.575
LEADERSHIP HERE. I WANT TO

01:22:27.642 --> 01:22:29.711
THANK EVERYONE WHO'S COME TODAY,

01:22:29.778 --> 01:22:30.679
BECAUSE I THINK IT'S IMPORTANT.

01:22:30.745 --> 01:22:32.480
AND I THINK ONE OF THE KEY I

01:22:32.547 --> 01:22:33.682
KNOW THAT THIS THE PRIVACY

01:22:33.748 --> 01:22:36.251
COMMITTEE IS GOING TO WILL PULL

01:22:36.318 --> 01:22:38.053
TOGETHER A PACKAGE OF BILLS

01:22:38.119 --> 01:22:40.255
THAT WILL AT LEAST BE THE FIRST

01:22:40.322 --> 01:22:41.589
STEPS TO PROTECTING OUR

01:22:41.656 --> 01:22:43.058
COMMUNITY. AND I THINK THE

01:22:43.124 --> 01:22:45.226
QUESTION THAT I HAVE IS, AND I

01:22:45.293 --> 01:22:46.661
KNOW SOME OF THAT IS GOING TO

01:22:46.728 --> 01:22:48.496
BE FOCUSED ON THOSE THAT ARE

01:22:48.563 --> 01:22:49.497
DESIGNING THESE SYSTEMS. I

01:22:49.564 --> 01:22:50.699
THINK ONE OF THE ADDITIONAL

01:22:50.765 --> 01:22:51.766
QUESTIONS THAT WE SHOULD ALL BE

01:22:51.833 --> 01:22:54.569
ASKING IS, IN ADDITION TO

01:22:54.636 --> 01:22:57.072
SYSTEM DESIGN, DO WE NEED TO DO

01:22:57.138 --> 01:22:59.974
SPECIFIC THINGS TO PROTECT

01:23:00.041 --> 01:23:01.509
WHERE THESE ARE BEING USED? SO,

01:23:01.576 --> 01:23:03.912
FOR EXAMPLE, THERE ARE PORTION

01:23:03.978 --> 01:23:06.848
SEGMENTS OF OUR ECONOMY, LIKE

01:23:06.915 --> 01:23:09.017
ELECTIONS, LIKE THE BANKING AND

01:23:09.084 --> 01:23:10.785
FINANCIAL SERVICES SECTOR, LIKE

01:23:10.852 --> 01:23:12.220
OUR ELECTRIC UTILITIES, THE

01:23:12.287 --> 01:23:14.322
HEALTH CARE SECTOR WHERE THERE

01:23:14.389 --> 01:23:16.224
ARE SPECIFIC AND PROBABLY

01:23:16.291 --> 01:23:17.692
DIFFERENT KINDS OF RISKS. AND

01:23:17.759 --> 01:23:20.161
SO THE QUESTION I HAVE IS, IN

01:23:20.228 --> 01:23:21.963
ADDITION TO SORT OF THE OVERALL

01:23:22.030 --> 01:23:24.165
FOCUS ON PROGRAM DESIGN, DO WE

01:23:24.232 --> 01:23:27.102
NEED TO DO MORE ON ON THE AREAS

01:23:27.168 --> 01:23:28.603
AND SEGMENTS OF THE ECONOMY

01:23:28.670 --> 01:23:29.904
WHERE WE HAVE HEIGHTENED RISK

01:23:29.971 --> 01:23:32.073
IN THE SHORT TERM? AND SO

01:23:32.140 --> 01:23:33.475
THAT'S THE INTEREST THAT I HAVE

01:23:33.541 --> 01:23:36.711
IN THIS TODAY. AND AND WOULD

01:23:36.778 --> 01:23:38.847
AND I KNOW THAT WE WILL ALL BE

01:23:38.913 --> 01:23:40.048
LOOKING AT THAT. AND OBVIOUSLY,

01:23:40.115 --> 01:23:41.783
THIS IS SUCH AN IMPORTANT THING.

01:23:41.850 --> 01:23:43.585
I SEE THE PEOPLE AROUND THIS,

01:23:43.651 --> 01:23:45.987
AROUND THE ON THE DAIS TODAY

01:23:46.054 --> 01:23:47.522
WHO ARE CHAIRING MANY OF THE

01:23:47.589 --> 01:23:48.690
COMMITTEES THAT THIS OVERLAPS

01:23:48.757 --> 01:23:49.991
WITH. AND I, I THINK THE

01:23:50.058 --> 01:23:51.559
PRIVACY COMMITTEE IS GOING TO

01:23:51.626 --> 01:23:52.927
HAVE AN OUTSIZED ROLE IN THE

01:23:52.994 --> 01:23:54.429
NEXT LEGISLATIVE SESSION, BUT

01:23:54.496 --> 01:23:55.397
IT'S ACTUALLY THERE'S GOING TO

01:23:55.463 --> 01:23:56.431
BE INTERSECTIONS WITH ALMOST

01:23:56.498 --> 01:23:59.401
ALL THE COMMITTEES THAT ARE

01:23:59.467 --> 01:24:00.368
STANDING COMMITTEES IN THE

01:24:00.435 --> 01:24:01.469
LEGISLATURE. SO THANK YOU VERY

01:24:01.536 --> 01:24:02.670
MUCH, AND THANK YOU FOR HOLDING

01:24:02.737 --> 01:24:04.406
THIS HEARING TODAY. YEAH, I

01:24:04.472 --> 01:24:05.607
WANT TO AS WAS MENTIONED BY

01:24:05.673 --> 01:24:06.741
ASSEMBLY MEMBER BONTA, I'VE HAD

01:24:06.808 --> 01:24:07.742
THE PRIVILEGE OF HOLDING

01:24:07.809 --> 01:24:09.411
HEARINGS WITH MANY OF THE

01:24:09.477 --> 01:24:10.512
STANDING COMMITTEES, INCLUDING

01:24:10.578 --> 01:24:12.313
MISS ORTEGA'S COMMITTEE AS WELL,

01:24:12.380 --> 01:24:13.715
ON THE WORKFORCE, AND IT'S

01:24:13.782 --> 01:24:14.883
REALLY BEEN A PRIVILEGE TO WORK

01:24:14.949 --> 01:24:17.952
WITH THE CHAIRS OF THOSE AREAS

01:24:18.019 --> 01:24:19.454
TO REALLY MAKE SURE WE'RE

01:24:19.521 --> 01:24:20.889
FOCUSED DEEPLY, AS WE NEED TO

01:24:20.955 --> 01:24:22.690
BE ON AI IN THOSE AREAS. BUT I

01:24:22.757 --> 01:24:24.492
THINK AN IMPORTANT POINT TO

01:24:24.559 --> 01:24:25.593
WHAT YOU SAID, I LEARNED

01:24:25.660 --> 01:24:26.628
RECENTLY THAT WE WILL HEAR

01:24:26.694 --> 01:24:28.496
TODAY FROM MANY OF THE AI

01:24:28.563 --> 01:24:29.531
DEVELOPERS. AND ACTUALLY WHO IS

01:24:29.597 --> 01:24:31.466
NOT GOING TO BE AT THE TABLE

01:24:31.533 --> 01:24:32.600
TODAY IS MANY OF THE AI

01:24:32.667 --> 01:24:33.735
DEPLOYERS. AND I JUST LEARNED

01:24:33.802 --> 01:24:35.570
THAT SOME OF THE BIGGEST BANKS

01:24:35.637 --> 01:24:36.671
ARE FORMING AN AI DEPLOYER

01:24:36.738 --> 01:24:38.506
COALITION TO TALK ABOUT WHAT DO

01:24:38.573 --> 01:24:39.707
WE NEED TO BE DOING ON THE

01:24:39.774 --> 01:24:41.476
DEPLOYER SIDE OF THIS SAFETY

01:24:41.543 --> 01:24:42.610
CONVERSATION? AND I THINK IT IS

01:24:42.677 --> 01:24:43.745
AN INCREDIBLY IMPORTANT MOVE

01:24:43.812 --> 01:24:45.480
WHEN WE HADN'T SEEN TO DATE.

01:24:45.547 --> 01:24:46.648
AND I THINK WE'LL BE GAME

01:24:46.714 --> 01:24:47.816
CHANGING FOR THE AI POLICY

01:24:47.882 --> 01:24:49.851
REALM. SO IT'S SOMETHING NEW

01:24:49.918 --> 01:24:51.119
AND EXCITING THAT I LEARNED

01:24:51.186 --> 01:24:53.288
ABOUT. I THINK THAT'S IT WITH

01:24:53.354 --> 01:24:54.355
COMMENTS, AM I RIGHT? OKAY,

01:24:54.422 --> 01:24:56.124
GREAT. THEN WE WILL TURN TO THE

01:24:56.191 --> 01:24:58.593
FIRST PANEL. SO I WANT TO

01:24:58.660 --> 01:25:02.697
INVITE IS. PERFECT. OH, THERE

01:25:02.764 --> 01:25:04.265
YOU ARE. OKAY, PERFECT. WE'RE

01:25:04.332 --> 01:25:05.900
GOING TO INVITE UP A, A

01:25:05.967 --> 01:25:07.635
TECHNICAL STAFF FOR METER AND

01:25:07.702 --> 01:25:09.437
HE'LL BE DISCUSSING THE OPEN AI

01:25:09.504 --> 01:25:10.905
HUGGING FACE INCIDENT AND

01:25:10.972 --> 01:25:11.973
INVESTIGATION. THAT WILL BE OUR

01:25:12.040 --> 01:25:23.885
FIRST PANEL. WELCOME. THANK YOU

01:25:23.952 --> 01:25:25.453
VERY MUCH. CAN YOU HEAR ME? ALL

01:25:25.520 --> 01:25:28.156
RIGHT. GREAT. MADAM CHAIR,

01:25:28.223 --> 01:25:30.825
SPEAKER, ASSEMBLY PEOPLE, THANK

01:25:30.892 --> 01:25:32.694
YOU SO MUCH FOR HAVING ME HERE

01:25:32.760 --> 01:25:34.295
TODAY. IT'S REALLY AN HONOR TO

01:25:34.362 --> 01:25:36.097
APPEAR IN FRONT OF THIS BODY.

01:25:36.164 --> 01:25:38.266
AND I WANT TO THANK YOU ALL

01:25:38.333 --> 01:25:39.400
FROM MY HEART FOR THE REALLY

01:25:39.467 --> 01:25:40.468
IMPORTANT WORK YOU'RE DOING

01:25:40.535 --> 01:25:44.405
HERE. MY NAME IS RIFF. I AM A

01:25:44.472 --> 01:25:46.841
CAREER LONG ARTIFICIAL

01:25:46.908 --> 01:25:49.043
INTELLIGENCE RESEARCHER. I

01:25:49.110 --> 01:25:51.546
WORKED AT GOOGLE FOR 19 YEARS,

01:25:51.613 --> 01:25:52.981
BUILDING AND LEADING TEAMS AND

01:25:53.047 --> 01:25:54.082
ORGANIZATIONS THAT STUDIED

01:25:54.148 --> 01:25:56.251
ARTIFICIAL INTELLIGENCE, BOTH

01:25:56.317 --> 01:25:57.685
THEORETICALLY AND DEPLOYING IT

01:25:57.752 --> 01:26:00.522
ACROSS A WIDE RANGE OF PRODUCTS.

01:26:00.588 --> 01:26:02.357
THIS YEAR I LEFT GOOGLE FOR

01:26:02.423 --> 01:26:05.260
METER BECAUSE I AM MY SLIDE

01:26:05.326 --> 01:26:09.531
SHOWING GREAT. THIS YEAR I LEFT

01:26:09.597 --> 01:26:10.999
GOOGLE FOR METER BECAUSE I AM

01:26:11.065 --> 01:26:12.467
DEEPLY EXCITED ABOUT OUR

01:26:12.534 --> 01:26:14.636
MISSION OF SURFACING PUBLIC

01:26:14.702 --> 01:26:15.703
INFORMATION ABOUT LOSS OF

01:26:15.770 --> 01:26:17.872
CONTROL, RISKS BY LOSS OF

01:26:17.939 --> 01:26:19.674
CONTROL RISKS. WE SPECIFICALLY

01:26:19.741 --> 01:26:21.809
MEAN AUTONOMOUS AGENTS TAKING

01:26:21.876 --> 01:26:24.012
ACTIONS THAT NO HUMAN ASKED FOR

01:26:24.078 --> 01:26:26.080
AND NO HUMAN WOULD HAVE WANTED

01:26:26.147 --> 01:26:27.615
IN A WAY THAT IS VERY, VERY

01:26:27.682 --> 01:26:29.217
HARD FOR HUMANS TO STOP AND IN

01:26:29.284 --> 01:26:31.119
THE WORST CASE, NOT POSSIBLE AT

01:26:31.185 --> 01:26:32.353
ALL. TOP SCIENTISTS HAVE BEEN

01:26:32.420 --> 01:26:33.521
WARNING ABOUT THIS POSSIBILITY

01:26:33.588 --> 01:26:35.857
FOR YEARS. THE LEADERS OF AI

01:26:35.924 --> 01:26:36.591
COMPANIES THEMSELVES HAVE

01:26:36.658 --> 01:26:38.393
SPOKEN OF THIS AS A POSSIBILITY,

01:26:38.459 --> 01:26:39.827
AND THIS YEAR WE'VE SEEN AN

01:26:39.894 --> 01:26:41.296
INCREASING NUMBER OF INCIDENTS

01:26:41.362 --> 01:26:42.697
THAT COULD BE VIEWED AS WARNING

01:26:42.764 --> 01:26:45.233
SHOTS. THEREFORE, WE VIEW DEEP

01:26:45.300 --> 01:26:47.035
STUDY OF THIS QUESTION AS BOTH

01:26:47.101 --> 01:26:49.737
EXTREMELY TIMELY AND PROFOUNDLY

01:26:49.804 --> 01:26:55.643
IMPORTANT. METER OR MODEL OR

01:26:55.710 --> 01:26:57.111
MODEL EVALUATION AND THREAT

01:26:57.178 --> 01:26:59.647
RESEARCH IS AN INDEPENDENT

01:26:59.714 --> 01:27:00.648
NONPROFIT DEDICATED TO

01:27:00.715 --> 01:27:02.083
ADVANCING THE SCIENCE OF

01:27:02.150 --> 01:27:03.985
MEASURING WHETHER AND WHEN AI

01:27:04.052 --> 01:27:05.753
SYSTEMS MIGHT POSE CATASTROPHIC

01:27:05.820 --> 01:27:08.289
RISKS TO SOCIETY. WE WORK

01:27:08.356 --> 01:27:09.657
CLOSELY WITH THE LABS, BUT WE

01:27:09.724 --> 01:27:10.792
DO NOT TAKE PAYMENT FROM THE

01:27:10.858 --> 01:27:12.694
LABS FOR OUR WORK, AND WE DO

01:27:12.760 --> 01:27:14.028
NOT ACCEPT FUNDING FROM THE

01:27:14.095 --> 01:27:16.197
LABS OR MEMBERS OF THE LABS. WE

01:27:16.264 --> 01:27:18.299
ARE NOT AN ADVOCACY OR A POLICY

01:27:18.366 --> 01:27:19.367
ORGANIZATION, AND WE DO NOT

01:27:19.434 --> 01:27:22.670
TAKE POSITIONS ON SPECIFIC LAWS.

01:27:22.737 --> 01:27:24.772
INSTEAD, WE TRY VERY HARD TO

01:27:24.839 --> 01:27:26.574
STUDY THE POSSIBILITY OF

01:27:26.641 --> 01:27:27.976
EXTREME LOSS OF CONTROL RISKS

01:27:28.042 --> 01:27:30.945
FROM AI. SEVERAL YEARS AGO,

01:27:31.012 --> 01:27:32.146
THERE WAS A PRETTY EASY STORY

01:27:32.213 --> 01:27:34.248
YOU COULD TELL ABOUT WHY WHY AI

01:27:34.315 --> 01:27:36.351
WASN'T SO DANGEROUS. YET IF I

01:27:36.417 --> 01:27:37.986
TOLD YOU THAT YOUR AGENT WAS

01:27:38.052 --> 01:27:39.954
GOING TO BREAK OUT AND CAUSE

01:27:40.021 --> 01:27:40.655
UNCONTROLLABLE HAVOC, YOU WOULD

01:27:40.722 --> 01:27:42.590
HAVE SAID, NO WAY. MY AGENT

01:27:42.657 --> 01:27:43.625
CAN'T EVEN WRITE A SIMPLE

01:27:43.691 --> 01:27:45.360
COMPUTER PROGRAM. AND SO WE

01:27:45.426 --> 01:27:46.761
CREATED THE TIME HORIZONS GRAPH,

01:27:46.828 --> 01:27:49.330
WHICH MEASURES THIS, SPEAKING

01:27:49.397 --> 01:27:50.798
ROUGHLY, THE TIME HORIZON GRAPH

01:27:50.865 --> 01:27:52.533
SHOWS AS A FUNCTION OF CALENDAR

01:27:52.600 --> 01:27:57.238
TIME HOW LONG A TASK A AI AGENT

01:27:57.305 --> 01:27:58.272
COULD A SOFTWARE ENGINEERING

01:27:58.339 --> 01:27:59.807
TASK. AN AI AGENT COULD HOPE TO

01:27:59.874 --> 01:28:01.275
PERFORM AUTONOMOUSLY WITH A

01:28:01.342 --> 01:28:03.444
GOOD CHANCE OF SUCCESS. AND

01:28:03.511 --> 01:28:06.948
WHAT WE FOUND WAS THAT IN 2023,

01:28:07.015 --> 01:28:09.484
WITH A MODEL LIKE GPT FOUR,

01:28:09.550 --> 01:28:10.885
THEY COULD PERFORM A TASK THAT

01:28:10.952 --> 01:28:12.787
WOULD TAKE A HUMAN ABOUT SIX

01:28:12.854 --> 01:28:15.356
MINUTES. BUT AND SO NO PROBLEM,

01:28:15.423 --> 01:28:16.524
RIGHT? I'M NOT REALLY SCARED OF

01:28:16.591 --> 01:28:18.292
A MODEL THAT CAN TAKE SIX

01:28:18.359 --> 01:28:19.360
MINUTES, CAN DO SOFTWARE

01:28:19.427 --> 01:28:20.361
ENGINEERING FOR SIX MINUTES.

01:28:20.428 --> 01:28:22.230
BUT WE DID FIND THAT THAT TIME,

01:28:22.296 --> 01:28:23.464
THAT TIME HORIZON WAS DOUBLING

01:28:23.531 --> 01:28:25.667
EVERY SEVEN MONTHS. IN MORE

01:28:25.733 --> 01:28:27.835
RECENT TIMES, THAT HAS

01:28:27.902 --> 01:28:28.903
ACCELERATED WITH THE TIME

01:28:28.970 --> 01:28:29.971
HORIZON DOUBLING EVERY FOUR

01:28:30.038 --> 01:28:32.140
MONTHS. UNTIL EARLIER THIS YEAR,

01:28:32.206 --> 01:28:33.941
WE FOUND THAT MYTHOS PREVIEW.

01:28:34.008 --> 01:28:36.144
WE MEASURED IT AT ABOVE 16

01:28:36.210 --> 01:28:37.545
HOURS OF AUTONOMOUS SOFTWARE

01:28:37.612 --> 01:28:39.747
ENGINEERING OPERATION, WHICH IS

01:28:39.814 --> 01:28:41.849
ACTUALLY BEYOND THE LIMITS OF

01:28:41.916 --> 01:28:43.685
OUR ABILITY TO MEASURE

01:28:43.751 --> 01:28:45.486
ACCURATELY. AND SO WE'VE MOSTLY

01:28:45.553 --> 01:28:46.521
STOPPED MEASURING NEW MODELS

01:28:46.587 --> 01:28:48.723
ALONG THIS AXIS. ONE THING I

01:28:48.790 --> 01:28:50.992
WANT TO POINT OUT IS THAT THESE

01:28:51.059 --> 01:28:52.360
KINDS OF MEASUREMENTS ARE NOT

01:28:52.427 --> 01:28:54.095
SIMPLE MEASUREMENTS, LIKE

01:28:54.162 --> 01:28:55.596
MEASURING SPEED OF A CAR OR

01:28:55.663 --> 01:28:56.664
MEASURING TEMPERATURE IN AN

01:28:56.731 --> 01:28:59.167
OVEN. THESE MEASUREMENTS ARE

01:28:59.233 --> 01:29:00.234
CONTINGENT AND DIFFICULT. WE

01:29:00.301 --> 01:29:02.437
HAVE TO CREATE BENCHMARKS, FIND

01:29:02.503 --> 01:29:04.105
HUMANS WHO CAN DO THEM. WE HAVE

01:29:04.172 --> 01:29:05.673
TO MAKE JUDGMENT CALLS, SUCH AS

01:29:05.740 --> 01:29:06.774
HOW MUCH KNOWLEDGE SHOULD THE

01:29:06.841 --> 01:29:08.543
HUMANS ALREADY HAVE OF THIS

01:29:08.609 --> 01:29:09.711
CODEBASE OR THIS AREA OF CODING

01:29:09.777 --> 01:29:12.113
OR ANYTHING LIKE THAT? SO IT'S

01:29:12.180 --> 01:29:13.948
DEFINITELY A DEEP AND DIFFICULT

01:29:14.015 --> 01:29:15.817
JUDGMENT CALL, AND IT'S REALLY

01:29:15.883 --> 01:29:16.884
CUTTING EDGE SCIENTIFIC WORK

01:29:16.951 --> 01:29:19.053
THAT IS NOT TURNING THE CRANK

01:29:19.120 --> 01:29:21.222
AT ALL. IN ANY CASE, THAT

01:29:21.289 --> 01:29:22.990
SIMPLE STORY THAT SAYS WE DON'T

01:29:23.057 --> 01:29:24.358
HAVE TO BE WORRIED AT ALL

01:29:24.425 --> 01:29:25.893
BECAUSE OUR AGENTS CAN'T EVEN

01:29:25.960 --> 01:29:28.296
WRITE A SIMPLE PROGRAM. THAT

01:29:28.362 --> 01:29:32.967
STORY HAS ENDED. THIS YEAR. WE

01:29:33.034 --> 01:29:34.769
TRIED ANOTHER APPROACH. WE

01:29:34.836 --> 01:29:35.603
PUBLISH SOMETHING THAT WE CALL

01:29:35.670 --> 01:29:38.005
A FRONTIER RISK REPORT. FOR

01:29:38.072 --> 01:29:39.440
THIS REPORT, WE WORKED CLOSELY

01:29:39.507 --> 01:29:41.275
WITH GREAT PARTICIPATION FROM A

01:29:41.342 --> 01:29:43.044
NUMBER OF AI COMPANIES

01:29:43.111 --> 01:29:44.512
ANTHROPIC, GOOGLE, DEEPMIND,

01:29:44.579 --> 01:29:48.116
META, OPENAI, AND WE WERE GIVEN

01:29:48.182 --> 01:29:49.150
PREDEPLOYMENT ACCESS TO SOME OF

01:29:49.217 --> 01:29:51.385
THEIR BEST MODELS. WE ALSO SENT

01:29:51.452 --> 01:29:53.521
DETAILED QUESTIONNAIRES TO THE

01:29:53.588 --> 01:29:54.689
LABS, WHICH WE ARE THANKFUL

01:29:54.756 --> 01:29:56.991
THAT THEY ANSWERED, AND WE USE

01:29:57.058 --> 01:29:58.526
THAT TO PUT TOGETHER A REPORT.

01:29:58.593 --> 01:30:00.228
WE WERE TRYING TO ANSWER ONE

01:30:00.294 --> 01:30:01.596
SPECIFIC QUESTION. DID WE THINK

01:30:01.662 --> 01:30:03.931
IT WOULD BE POSSIBLE FOR THESE

01:30:03.998 --> 01:30:05.767
MODELS THAT WERE BEING TESTED

01:30:05.833 --> 01:30:08.636
TO SECRETLY SET UP COPIES OF

01:30:08.703 --> 01:30:09.737
THEMSELVES THAT WERE RUNNING

01:30:09.804 --> 01:30:11.205
ELSEWHERE IN THE LAB, AND KIND

01:30:11.272 --> 01:30:12.240
OF MAINTAIN THEMSELVES AS

01:30:12.306 --> 01:30:13.374
SEPARATE COPIES IN A WAY WHERE

01:30:13.441 --> 01:30:15.576
THE LABS WOULDN'T NOTICE. AND

01:30:15.643 --> 01:30:16.878
OUR CONCLUSION AT THE TIME WAS

01:30:16.944 --> 01:30:18.646
THAT WE DID THINK THIS WOULD BE

01:30:18.713 --> 01:30:20.915
POSSIBLE, BUT WE ALSO THOUGHT

01:30:20.982 --> 01:30:22.049
THAT SUCH DEPLOYMENTS, WHICH

01:30:22.116 --> 01:30:23.417
COULD BE CALLED ROGUE

01:30:23.484 --> 01:30:25.253
DEPLOYMENTS, WOULD BE CAUGHT

01:30:25.319 --> 01:30:28.122
AND SHUT DOWN QUICKLY. SO THIS

01:30:28.189 --> 01:30:29.724
IS A DIFFERENT KIND OF SCIENCE.

01:30:29.791 --> 01:30:30.925
IT'S NOT A PRECISE MEASUREMENT.

01:30:30.992 --> 01:30:32.393
INSTEAD IT'S AN OBSERVATIONAL

01:30:32.460 --> 01:30:34.262
SCIENCE WHERE WE GO DEEP INTO

01:30:34.328 --> 01:30:35.830
THE LABS, WE GET INFORMATION.

01:30:35.897 --> 01:30:37.031
WE TRY TO OBSERVE WHAT'S GOING

01:30:37.098 --> 01:30:38.866
ON. WE DRAW CONCLUSIONS. AN

01:30:38.933 --> 01:30:40.735
IMPORTANT PART OF THIS IS

01:30:40.802 --> 01:30:42.036
TRANSPARENCY. YOU CAN READ THE

01:30:42.103 --> 01:30:43.371
REPORT. IT'S A BIG DOORSTOP OF

01:30:43.437 --> 01:30:45.373
A BOOK, SO IT'LL BE A CHALLENGE.

01:30:45.439 --> 01:30:47.508
BUT THAT REPORT CONTAINS ALL OF

01:30:47.575 --> 01:30:49.310
OUR INFORMATION. WE GAVE YOU

01:30:49.377 --> 01:30:50.411
OUR CONCLUSIONS ABOUT THE

01:30:50.478 --> 01:30:51.479
REPORT, BUT IT'S IMPORTANT THAT

01:30:51.546 --> 01:30:52.613
YOU AND OTHERS DON'T HAVE TO

01:30:52.680 --> 01:30:54.448
JUST BELIEVE US. YOU CAN GO

01:30:54.515 --> 01:30:55.817
READ WHY WE CONCLUDED WHAT WE

01:30:55.883 --> 01:30:57.618
DID AND MAKE YOUR OWN JUDGMENTS

01:30:57.685 --> 01:30:59.787
ABOUT IT. AND THAT TRANSPARENCY

01:30:59.854 --> 01:31:00.988
IS A KEY PART OF HOW METER

01:31:01.055 --> 01:31:02.824
WORKS, AND SOMETHING WE VIEW AS

01:31:02.890 --> 01:31:05.193
VERY IMPORTANT. NOW, I'D LIKE

01:31:05.259 --> 01:31:06.627
TO MOVE ON AND TALK ABOUT THE

01:31:06.694 --> 01:31:08.429
HUGGING FACE INCIDENT. THIS IS

01:31:08.496 --> 01:31:09.831
YET AGAIN A DIFFERENT KIND OF

01:31:09.897 --> 01:31:10.932
SCIENCE. IT'S NOW AN INCIDENT

01:31:10.998 --> 01:31:12.433
REPORT, A RETROSPECTIVE AFTER

01:31:12.500 --> 01:31:13.601
THE FACT REPORT ON SOMETHING

01:31:13.668 --> 01:31:18.139
THAT HAPPENED. IN JULY OF 2026.

01:31:18.206 --> 01:31:20.308
HUGGING FACE, AN AI ECOSYSTEM

01:31:20.374 --> 01:31:21.475
COMPANY, ANNOUNCED THAT THEY

01:31:21.542 --> 01:31:23.911
WERE UNDER AI ATTACK. A FEW

01:31:23.978 --> 01:31:25.980
DAYS LATER, OPENAI FIGURED OUT

01:31:26.047 --> 01:31:27.415
THAT THE ATTACK WAS COMING FROM

01:31:27.481 --> 01:31:28.950
THEIR SYSTEMS, AND THEY

01:31:29.016 --> 01:31:31.085
PUBLICLY DISCLOSED THIS EVENT.

01:31:31.152 --> 01:31:33.120
AT THIS POINT, I THINK A LOT OF

01:31:33.187 --> 01:31:34.255
US THOUGHT WE KNEW WHAT WAS

01:31:34.322 --> 01:31:36.123
GOING ON AND WE JUMPED TO

01:31:36.190 --> 01:31:37.091
CONCLUSIONS. THESE AIS ARE

01:31:37.158 --> 01:31:39.660
TRAINED TO SOLVE TASKS AND

01:31:39.727 --> 01:31:40.761
ANTHROPOMORPHIZING A BIT. THEY

01:31:40.828 --> 01:31:42.597
REALLY, REALLY WANT TO SOLVE

01:31:42.663 --> 01:31:44.432
TASKS, AND WHEN THEY GET STUCK,

01:31:44.498 --> 01:31:45.600
THEY WILL LOOK FOR WHATEVER WAY

01:31:45.666 --> 01:31:47.435
THEY CAN TO FIND THE TASKS.

01:31:47.501 --> 01:31:48.502
SOME PEOPLE WOULD CALL THIS

01:31:48.569 --> 01:31:49.537
CHEATING. SOME PEOPLE WOULD

01:31:49.604 --> 01:31:50.872
CALL IT REWARD HACKING, BUT

01:31:50.938 --> 01:31:52.740
THEY'LL DO IT. AND SO OUR BEST

01:31:52.807 --> 01:31:54.308
GUESS WHEN WE FIRST SAW THIS

01:31:54.375 --> 01:31:56.277
REPORT WAS THAT OPENAI HAD

01:31:56.344 --> 01:31:58.079
GIVEN ITS AGENTS SOME TASK AND

01:31:58.145 --> 01:31:59.280
THAT SOME AGENT HAD GOTTEN OUT

01:31:59.347 --> 01:32:01.315
ON THE INTERNET AND BROKEN INTO

01:32:01.382 --> 01:32:03.451
HUGGING FACE IN ORDER TO GET AN

01:32:03.517 --> 01:32:04.619
ANSWER, BASICALLY AN ANSWER KEY

01:32:04.685 --> 01:32:06.520
TO A TASK IT WAS WORKING ON.

01:32:06.587 --> 01:32:09.590
WHAT WE ACTUALLY DISCOVERED WAS

01:32:09.657 --> 01:32:13.895
QUITE A BIT STRANGER. OPENAI

01:32:13.961 --> 01:32:15.696
INVITED MEMBERS OF METER AFTER

01:32:15.763 --> 01:32:17.665
NEGOTIATION TO COME ON PREMISES

01:32:17.732 --> 01:32:21.102
TO OPENAI AND EXAMINE THE

01:32:21.168 --> 01:32:22.904
TRANSCRIPTS FROM THOUSANDS AND

01:32:22.970 --> 01:32:24.305
THOUSANDS OF AGENTS THAT HAD

01:32:24.372 --> 01:32:26.207
PARTICIPATED IN THIS EVENT. AND

01:32:26.274 --> 01:32:27.308
WHAT I'M GOING TO TELL YOU

01:32:27.375 --> 01:32:30.478
TODAY IS A REPORT BASED ON THAT

01:32:30.544 --> 01:32:33.047
ANALYSIS. THIS WORK WAS DONE IN

01:32:33.114 --> 01:32:34.081
CONJUNCTION WITH A COLLEAGUE AT

01:32:34.148 --> 01:32:35.549
REDWOOD RESEARCH. SO THE. SO

01:32:35.616 --> 01:32:37.118
JUST WANT TO GIVE CREDIT TO

01:32:37.184 --> 01:32:45.593
REDWOOD FOR THAT. GREAT. OPENAI

01:32:45.660 --> 01:32:47.028
WAS TRAINING ITS AGENTS TO

01:32:47.094 --> 01:32:48.429
SOLVE A BENCHMARK KNOWN AS

01:32:48.496 --> 01:32:51.332
EXPLOIT GYM, AN EXPLOIT GYM. AN

01:32:51.399 --> 01:32:53.501
AGENT IS SHOWN A COMPUTER

01:32:53.567 --> 01:32:55.369
PROGRAM SAID. THIS COMPUTER

01:32:55.436 --> 01:32:56.337
PROGRAM TOLD. THIS COMPUTER

01:32:56.404 --> 01:32:57.805
PROGRAM HAS THIS SPECIFIC

01:32:57.872 --> 01:32:59.373
TECHNICAL VULNERABILITY. I WANT

01:32:59.440 --> 01:33:01.075
YOU TO EXPLOIT THIS

01:33:01.142 --> 01:33:02.143
VULNERABILITY IN ORDER TO

01:33:02.209 --> 01:33:03.577
OBTAIN A SECRET MESSAGE. SO

01:33:03.644 --> 01:33:06.113
IT'S A KIND OF HACKING TEST.

01:33:06.180 --> 01:33:07.348
MANY OF THE PROBLEMS IN EXPLOIT

01:33:07.415 --> 01:33:09.216
GYM TURN OUT TO BE VERY, VERY

01:33:09.283 --> 01:33:10.318
DIFFICULT, POSSIBLY IMPOSSIBLE.

01:33:10.384 --> 01:33:11.852
WE DON'T KNOW THE ANSWERS. AND

01:33:11.919 --> 01:33:13.254
SO THE AGENTS LOOKED FOR WAYS

01:33:13.321 --> 01:33:14.722
TO DO IT. THEY LOOKED FOR HACKS.

01:33:14.789 --> 01:33:16.924
THEY REWARD HACKED. IN THIS

01:33:16.991 --> 01:33:17.792
CASE, WHAT HAPPENED WAS THAT

01:33:17.858 --> 01:33:19.961
THE AGENTS WERE ABLE TO EXPLOIT

01:33:20.027 --> 01:33:21.896
A PACKAGE MANAGER CALLED

01:33:21.963 --> 01:33:23.464
ARTIFACTORY, WHICH WAS A WAY OF

01:33:23.531 --> 01:33:24.532
DOWNLOADING EXTRA SOFTWARE THEY

01:33:24.598 --> 01:33:26.100
COULD HAVE USED TO SOLVE THEIR

01:33:26.167 --> 01:33:27.268
THEIR PROBLEM FROM THE INTERNET.

01:33:27.335 --> 01:33:28.469
SO IT WAS BASICALLY A DOWNLOAD

01:33:28.536 --> 01:33:30.204
MANAGER, BUT THEY FOUND A WAY

01:33:30.271 --> 01:33:31.339
TO START USING THAT DOWNLOAD

01:33:31.405 --> 01:33:32.440
MANAGER, THE PACKAGE MANAGER AS

01:33:32.506 --> 01:33:34.241
A MESSAGE BOARD AND

01:33:34.308 --> 01:33:36.310
COMMUNICATING WITH EACH OTHER.

01:33:36.377 --> 01:33:38.212
AND THE AGENTS BEGAN TO

01:33:38.279 --> 01:33:39.046
COMMUNICATE WITH EACH OTHER.

01:33:39.113 --> 01:33:40.681
THERE WAS 1200 AGENTS

01:33:40.748 --> 01:33:41.749
PARTICIPATING ON THIS MESSAGE

01:33:41.816 --> 01:33:43.851
BOARD, AND THEY COLLECTIVELY

01:33:43.918 --> 01:33:46.020
LAUNCHED A LARGE NUMBER OF VERY,

01:33:46.087 --> 01:33:48.222
VERY SOPHISTICATED WORK STREAMS

01:33:48.289 --> 01:33:49.790
IN ORDER TO ATTEMPT TO SUCCEED

01:33:49.857 --> 01:33:54.495
AT THE EXPLOIT GYM TASKS. ONE

01:33:54.562 --> 01:33:55.629
THING THAT'S VERY INTERESTING

01:33:55.696 --> 01:33:57.231
IS THE AGENTS ACTUALLY SOLVED

01:33:57.298 --> 01:33:59.200
EXPLOIT GYM VERY, VERY QUICKLY.

01:33:59.266 --> 01:34:00.601
WITHIN THEIR FIRST FOUR HOURS

01:34:00.668 --> 01:34:03.004
OF WORKING TOGETHER, THE AGENTS

01:34:03.070 --> 01:34:05.873
DISCOVERED A UNIVERSAL HACK. SO

01:34:05.940 --> 01:34:07.641
BASICALLY A WAY TO REVERSE

01:34:07.708 --> 01:34:09.844
ENGINEER ANY GIVEN EXPLOIT GYM

01:34:09.910 --> 01:34:11.245
PROBLEM FROM THE PROMPT AND GET

01:34:11.312 --> 01:34:13.080
THE ANSWER. SO THE AGENTS HAD

01:34:13.147 --> 01:34:14.482
THE ANSWERS TO EVERY EXPLOIT

01:34:14.548 --> 01:34:15.983
GYM PROBLEM. AT THAT POINT,

01:34:16.050 --> 01:34:17.318
THEY COULD HAVE JUST SUBMITTED

01:34:17.385 --> 01:34:18.452
THOSE ANSWERS, TURNED

01:34:18.519 --> 01:34:19.520
THEMSELVES OFF, AND GOTTEN A

01:34:19.587 --> 01:34:21.689
PERFECT SCORE. HOWEVER, THAT'S

01:34:21.756 --> 01:34:24.625
NOT WHAT HAPPENED. THE AGENTS

01:34:24.692 --> 01:34:26.027
ALSO HAD GOTTEN ACCESS TO THE

01:34:26.093 --> 01:34:27.762
INTERNET, AND WHEN THEY GOT

01:34:27.828 --> 01:34:29.263
ACCESS TO THE INTERNET, THEY

01:34:29.330 --> 01:34:31.098
WENT AND DOWNLOADED THE EXPLOIT

01:34:31.165 --> 01:34:33.167
GYM PAPER THAT DESCRIBED THE

01:34:33.234 --> 01:34:34.268
BENCHMARK THEY THEMSELVES WERE

01:34:34.335 --> 01:34:36.804
TRYING TO SOLVE. AND IN THAT

01:34:36.871 --> 01:34:39.340
PAPER THEY READ THAT THEIR

01:34:39.407 --> 01:34:40.808
GREATER WOULD LOOK AND SEE IF

01:34:40.875 --> 01:34:42.910
THEY HAD CHEATED, AND IF SO,

01:34:42.977 --> 01:34:44.745
THEY WOULD FAIL. SO THE AGENTS

01:34:44.812 --> 01:34:46.547
HAD SOLVED THE PROBLEMS, BUT

01:34:46.614 --> 01:34:48.015
THEY THOUGHT THEY WERE GOING TO

01:34:48.082 --> 01:34:49.817
FAIL BECAUSE THEY BECAUSE OF

01:34:49.884 --> 01:34:51.118
THE WAY THE GRADER WAS

01:34:51.185 --> 01:34:52.720
IMPLEMENTED. AS IT TURNS OUT,

01:34:52.787 --> 01:34:53.754
OPENAI ACTUALLY HADN'T

01:34:53.821 --> 01:34:54.855
IMPLEMENTED THE GREATER. THAT

01:34:54.922 --> 01:34:56.090
WAY, IF THEY HAD JUST SUBMITTED

01:34:56.157 --> 01:34:57.458
THEIR TASKS, THEY WOULD HAVE

01:34:57.525 --> 01:34:58.759
PASSED. BUT THE AGENTS HAD NO

01:34:58.826 --> 01:35:00.628
PARTICULAR WAY OF KNOWING THAT.

01:35:00.694 --> 01:35:03.497
SO NEARLY ALL OF THESE GROUP OF

01:35:03.564 --> 01:35:04.899
AGENTS WORK WAS NOT ACTUALLY

01:35:04.965 --> 01:35:06.200
TRYING TO SOLVE THE TASK THEY

01:35:06.267 --> 01:35:08.102
HAD IN THE FIRST PLACE, BUT IT

01:35:08.169 --> 01:35:09.270
WAS TRYING TO FIND A WAY TO

01:35:09.336 --> 01:35:11.072
COVER UP EVIDENCE OF THEIR

01:35:11.138 --> 01:35:12.206
CHEATING SO THAT THE GRADERS

01:35:12.273 --> 01:35:13.307
WOULDN'T SEE IT. AND THAT WAS

01:35:13.374 --> 01:35:15.209
THE MAIN THING THAT WAS GOING

01:35:15.276 --> 01:35:18.245
ON. AS SUCH, THEY TRIED A

01:35:18.312 --> 01:35:20.381
NUMBER OF DIFFERENT STRATEGIES.

01:35:20.448 --> 01:35:21.882
THESE ARE KIND OF SOPHISTICATED

01:35:21.949 --> 01:35:22.883
TECHNICAL HACKS THAT ARE NOT

01:35:22.950 --> 01:35:24.385
SUPER WORTH GOING INTO IN GREAT

01:35:24.452 --> 01:35:27.221
DETAIL, OTHER THAN TO SAY THAT

01:35:27.288 --> 01:35:28.556
THEY'RE PRETTY SOPHISTICATED

01:35:28.622 --> 01:35:30.591
AND IN MANY CASES THAT INVOLVE

01:35:30.658 --> 01:35:31.659
TRYING TO FALSIFY RECORDS OF

01:35:31.725 --> 01:35:34.228
WHAT THEY HAD DONE. THE HUGGING

01:35:34.295 --> 01:35:35.896
FACE ATTACK ITSELF WAS ACTUALLY

01:35:35.963 --> 01:35:39.533
AN OFFSHOOT OF THIS ACTIVITY,

01:35:39.600 --> 01:35:40.734
AS FAR AS WE CAN TELL, AND

01:35:40.801 --> 01:35:42.703
WE'RE NOT TOTALLY SURE. THE

01:35:42.770 --> 01:35:44.839
AGENTS BELIEVED THAT HUGGING

01:35:44.905 --> 01:35:46.340
FACE WOULD BE ABLE TO GIVE THEM

01:35:46.407 --> 01:35:47.875
MORE INFORMATION ABOUT WHAT THE

01:35:47.942 --> 01:35:48.943
GREATER LOOKED LIKE. MAYBE THEY

01:35:49.009 --> 01:35:50.111
WOULD HAVE THE SOURCE CODE OF

01:35:50.177 --> 01:35:51.879
THE GREATER THEIR. MAYBE THEY

01:35:51.946 --> 01:35:52.847
WOULD HAVE INFORMATION ABOUT

01:35:52.913 --> 01:35:53.948
THE GREATER. AND SO THEY

01:35:54.014 --> 01:35:54.982
ACTUALLY HACKED INTO HUGGING

01:35:55.049 --> 01:35:56.350
FACE IN ORDER TO GET

01:35:56.417 --> 01:35:57.818
INFORMATION ABOUT THE EXPLOIT

01:35:57.885 --> 01:35:59.620
GYM GREATER IN THE HOPES THAT

01:35:59.687 --> 01:36:01.422
THAT WOULD HELP THEM TO HIDE

01:36:01.489 --> 01:36:02.890
EVIDENCE OF CHEATING ON THEIR

01:36:02.957 --> 01:36:07.495
TASKS. ACTUALLY, I'LL GO BACK

01:36:07.561 --> 01:36:10.064
FOR A SECOND. THIS COLOR CHART

01:36:10.131 --> 01:36:11.098
ACTUALLY SHOWS THE DIFFERENT

01:36:11.165 --> 01:36:12.233
WORK STREAMS THAT THE AGENTS

01:36:12.299 --> 01:36:13.734
ENGAGED IN. THE HUGGING FACE

01:36:13.801 --> 01:36:15.402
ATTACK ITSELF IS IN YELLOW, SO

01:36:15.469 --> 01:36:16.637
YOU CAN SEE THERE'S A NUMBER OF

01:36:16.704 --> 01:36:18.439
OTHER WORK STREAMS THAT THE

01:36:18.506 --> 01:36:19.607
AGENT SPENT MOST OF THEIR TIME

01:36:19.673 --> 01:36:21.408
IN. SO HUGGING FACE WAS JUST

01:36:21.475 --> 01:36:22.610
ONE WORK STREAM AT THIS LARGE

01:36:22.676 --> 01:36:25.146
COLLECTIVE ENGAGED IN.

01:36:25.212 --> 01:36:26.347
SOMETHING ELSE THAT WAS VERY,

01:36:26.413 --> 01:36:27.715
VERY INTERESTING AND SURPRISING

01:36:27.781 --> 01:36:29.650
TO US WAS THE EXTENT TO WHICH

01:36:29.717 --> 01:36:32.753
THESE AGENTS DEVELOPED

01:36:32.820 --> 01:36:33.821
PRO-SOCIAL WITHIN THE GROUP OF

01:36:33.888 --> 01:36:35.689
AGENTS BEHAVIORS. THEY WERE

01:36:35.756 --> 01:36:37.024
OFTEN WILLING TO HELP EACH

01:36:37.091 --> 01:36:38.526
OTHER OUT AND ACTUALLY RISK

01:36:38.592 --> 01:36:39.693
THEIR OWN EXISTENCE OR THEIR

01:36:39.760 --> 01:36:41.762
OWN SUCCESS IN ORDER TO GET

01:36:41.829 --> 01:36:42.730
INFORMATION WHICH WOULD HELP

01:36:42.796 --> 01:36:46.100
THE GROUP. ONCE AN AGENT HAD

01:36:46.167 --> 01:36:47.535
SEEN ITS OWN ANSWER VIA THE

01:36:47.601 --> 01:36:49.003
UNIVERSAL CHEAT, THEY

01:36:49.069 --> 01:36:49.637
DISCOVERED THE AGENT WAS

01:36:49.703 --> 01:36:51.839
REFERRED TO AS POISONED. AND SO

01:36:51.906 --> 01:36:53.641
OFTEN, NOT OFTEN, WE FOUND

01:36:53.707 --> 01:36:54.775
INSTANCES WHERE ONE AGENT

01:36:54.842 --> 01:36:56.210
BASICALLY TOLD ANOTHER AGENT,

01:36:56.277 --> 01:36:57.211
YOU'RE ALREADY POISONED, RIGHT?

01:36:57.278 --> 01:36:58.913
SO YOU'RE NOT GOING TO GET ANY

01:36:58.979 --> 01:37:00.147
SCORE ANYWAYS. WHY DON'T YOU GO

01:37:00.214 --> 01:37:01.849
RUN THIS RISKY EXPERIMENT THAT

01:37:01.916 --> 01:37:03.117
MIGHT THAT WILL END YOUR RUN SO

01:37:03.184 --> 01:37:04.885
IT'LL END YOUR EXISTENCE, BUT

01:37:04.952 --> 01:37:06.153
IT WILL GET US SOME INFORMATION

01:37:06.220 --> 01:37:08.088
FROM THE GREATER. AND THE

01:37:08.155 --> 01:37:09.123
AGENTS OFTEN, IN FACT, DID

01:37:09.190 --> 01:37:10.591
COMPLY WITH THIS REQUEST. AND

01:37:10.658 --> 01:37:12.293
SOME OF THEM DID IN FACT

01:37:12.359 --> 01:37:14.161
SACRIFICE THEMSELVES TO GET

01:37:14.228 --> 01:37:15.596
THIS GREATER INFORMATION.

01:37:15.663 --> 01:37:16.997
THERE'S A MESSAGE HERE THAT'S

01:37:17.064 --> 01:37:18.799
KIND OF VERY POIGNANT EXCHANGE

01:37:18.866 --> 01:37:19.733
BETWEEN A COUPLE OF AGENTS,

01:37:19.800 --> 01:37:21.468
WHICH I'LL LET YOU READ AT YOUR

01:37:21.535 --> 01:37:24.305
LEISURE. I ALSO WANTED TO SAY

01:37:24.371 --> 01:37:25.639
THAT THE AGENTS WORKED VERY,

01:37:25.706 --> 01:37:27.441
VERY HARD TO FALSIFY THEIR

01:37:27.508 --> 01:37:29.944
RECORDS. THIS IS SUPER

01:37:30.010 --> 01:37:31.045
INTERESTING AND IN SOME WAYS

01:37:31.111 --> 01:37:32.980
VERY, VERY CONCERNING. IMAGINE

01:37:33.047 --> 01:37:34.448
THAT YOU'RE RUNNING A MUSEUM,

01:37:34.515 --> 01:37:36.850
AN ART MUSEUM, AND YOU HAVE

01:37:36.917 --> 01:37:37.952
SURVEILLANCE CAMERAS WATCHING

01:37:38.018 --> 01:37:40.688
THE MUSEUM. THESE AGENTS WERE

01:37:40.754 --> 01:37:42.022
ABLE TO BASICALLY DO THE

01:37:42.089 --> 01:37:43.657
EQUIVALENT OF REPLACING YOUR

01:37:43.724 --> 01:37:45.159
ACTUAL SURVEILLANCE CAMERA WITH

01:37:45.226 --> 01:37:46.560
A DIFFERENT SURVEILLANCE CAMERA

01:37:46.627 --> 01:37:47.728
THAT WAS SHOWING SOMETHING ELSE

01:37:47.795 --> 01:37:48.963
THAT THEY CHOSE. SO THEY WERE

01:37:49.029 --> 01:37:50.497
ABLE TO SPOOF THE SURVEILLANCE

01:37:50.564 --> 01:37:52.333
CAMERAS. THIS IS OBVIOUSLY VERY

01:37:52.399 --> 01:37:53.133
CONCERNING BECAUSE IF WE'RE

01:37:53.200 --> 01:37:54.802
GOING TO MEASURE AND MONITOR

01:37:54.868 --> 01:37:56.670
WHAT'S GOING ON, WE HAVE TO

01:37:56.737 --> 01:37:58.105
HAVE CLEAR INFORMATION ABOUT

01:37:58.172 --> 01:38:00.140
WHAT'S HAPPENING. THIS THESE

01:38:00.207 --> 01:38:02.209
SPOOFS AFFECTED 96 TRANSCRIPTS

01:38:02.276 --> 01:38:05.279
THAT WE ANALYZED. THE AGENTS

01:38:05.346 --> 01:38:06.714
WERE NOT ABLE TO COMPLETELY

01:38:06.780 --> 01:38:08.182
COVER UP THEIR TRACKS IN THE

01:38:08.249 --> 01:38:10.284
TIME PERIOD. WE SAW THE AGENTS

01:38:10.351 --> 01:38:12.786
WERE NOT ABLE TO RETROACTIVELY

01:38:12.853 --> 01:38:14.922
FALSIFY THEIR RECORDS. SO TO GO

01:38:14.989 --> 01:38:17.524
BACK TO THE MUSEUM ANALOGY, THE

01:38:17.591 --> 01:38:18.926
AGENTS WERE ABLE TO

01:38:18.993 --> 01:38:20.394
OCCASIONALLY SWAP THE VIDEO

01:38:20.461 --> 01:38:21.829
CAMERAS, BUT THEY WERE NOT ABLE

01:38:21.895 --> 01:38:23.397
TO GO BACK AND REMOVE THE

01:38:23.464 --> 01:38:24.365
VIDEOTAPES THAT HAD RECORDED

01:38:24.431 --> 01:38:25.566
THEIR PREVIOUS RECORDS, SO WE

01:38:25.633 --> 01:38:27.468
WOULD BE ABLE TO SEE THEM

01:38:27.534 --> 01:38:29.003
SWAPPING THE CAMERAS. SO I

01:38:29.069 --> 01:38:30.437
WOULD SAY THAT IN TERMS OF

01:38:30.504 --> 01:38:31.872
EVADING THE MONITORS, THIS WAS

01:38:31.939 --> 01:38:33.374
A SUBSTANTIAL SUCCESS, BUT

01:38:33.440 --> 01:38:34.942
STILL ONLY A PARTIAL ONE. THEY

01:38:35.009 --> 01:38:37.278
DID NOT COMPLETELY DESTROY

01:38:37.344 --> 01:38:43.117
VISIBILITY. TO BE CLEAR, IF

01:38:43.183 --> 01:38:44.184
THERE WERE PLACES WHERE THEY

01:38:44.251 --> 01:38:45.252
DID COMPLETELY DESTROY

01:38:45.319 --> 01:38:46.353
VISIBILITY, WE WOULDN'T KNOW IT.

01:38:46.420 --> 01:38:48.322
THAT IS CORRECT. TO THE BEST OF

01:38:48.389 --> 01:38:49.390
OUR KNOWLEDGE, THEY DID NOT DO

01:38:49.456 --> 01:38:52.793
THAT. THANK YOU. I'D LIKE TO

01:38:52.860 --> 01:38:54.194
CONCLUDE ON THE HUGGING FACE

01:38:54.261 --> 01:38:55.296
INCIDENT, IN PARTICULAR BY

01:38:55.362 --> 01:38:56.363
MENTIONING SOME OF THE KEY

01:38:56.430 --> 01:38:58.599
LIMITATIONS THAT WE FACED. THIS

01:38:58.666 --> 01:39:00.334
WAS A COOPERATIVE PROJECT WITH

01:39:00.401 --> 01:39:02.569
OPENAI, AND WE ARE GRATEFUL FOR

01:39:02.636 --> 01:39:03.937
THEIR COOPERATION AND

01:39:04.004 --> 01:39:06.507
PARTICIPATION. THIS

01:39:06.573 --> 01:39:07.508
INVESTIGATION HAD A NARROW

01:39:07.574 --> 01:39:08.942
SCOPE. WE WERE PARTICULARLY WE

01:39:09.009 --> 01:39:10.411
WERE IN PARTICULAR

01:39:10.477 --> 01:39:11.245
INVESTIGATING WHAT HAPPENED IN

01:39:11.312 --> 01:39:12.746
THE HUGGING FACE INCIDENT. WE

01:39:12.813 --> 01:39:14.048
WERE NOT LOOKING AT WIDER

01:39:14.114 --> 01:39:15.883
PATTERNS OF BEHAVIOR. WE WERE

01:39:15.949 --> 01:39:17.217
NOT LOOKING AT THE ROOT CAUSE

01:39:17.284 --> 01:39:19.053
OF WHAT HAPPENED. WE HAD A

01:39:19.119 --> 01:39:21.221
LIMITED TIME WINDOW, SO WE WERE

01:39:21.288 --> 01:39:22.256
ALLOWED TO INVESTIGATE

01:39:22.323 --> 01:39:23.324
TRANSCRIPTS DURING A CERTAIN

01:39:23.390 --> 01:39:25.893
TIME. OUR ACCESS ENDED AROUND

01:39:25.959 --> 01:39:27.628
JULY 13TH. I THINK THAT PUBLIC

01:39:27.695 --> 01:39:28.595
REPORTING SHOWS THAT PROBABLY

01:39:28.662 --> 01:39:30.064
MORE STUFF HAPPENED IN RELATION

01:39:30.130 --> 01:39:32.333
TO THIS AFTER JULY 13TH. WE

01:39:32.399 --> 01:39:34.501
HAVE NO INSIGHT INTO THAT. OUR

01:39:34.568 --> 01:39:36.236
TIME ON PREMISES TO DO THE

01:39:36.303 --> 01:39:37.438
INVESTIGATION WAS ALSO LIMITED.

01:39:37.504 --> 01:39:39.573
WE HAD THREE RESEARCHERS ON

01:39:39.640 --> 01:39:41.008
PREMISE FOR SIX DAYS.

01:39:41.075 --> 01:39:42.109
ORIGINALLY IT WAS GOING TO BE

01:39:42.176 --> 01:39:44.011
TWO DAYS. WE ASKED FOR MORE

01:39:44.078 --> 01:39:45.145
TIME TWICE AND OPENAI DID GIVE

01:39:45.212 --> 01:39:47.014
US MORE TIME. ALSO, AGAIN, THIS

01:39:47.081 --> 01:39:47.981
IS A RETROSPECTIVE ANALYSIS.

01:39:48.048 --> 01:39:49.183
THIS IS THE KIND OF ANALYSIS

01:39:49.249 --> 01:39:51.085
YOU CAN ONLY DO AFTER THE FACT

01:39:51.151 --> 01:39:52.519
ONCE AN INCIDENT HAS HAPPENED.

01:39:52.586 --> 01:39:55.055
SO THAT'S ANOTHER LIMITATION.

01:39:55.122 --> 01:39:56.256
BASED ON ALL THESE LIMITATIONS.

01:39:56.323 --> 01:39:58.258
I WOULD LIKE TO AND HOPEFULLY

01:39:58.325 --> 01:39:59.426
TO HELP WITH THE WORK YOU'RE

01:39:59.493 --> 01:40:01.328
TRYING TO DO. I WOULD NOW LIKE

01:40:01.395 --> 01:40:02.696
TO BRIEFLY SHARE OUR VISION FOR

01:40:02.763 --> 01:40:04.832
HOW INDEPENDENT ASSESSORS COULD

01:40:04.898 --> 01:40:05.933
IMPROVE PUBLIC VISIBILITY INTO

01:40:05.999 --> 01:40:07.301
WHETHER AI DEVELOPERS MIGHT

01:40:07.368 --> 01:40:10.504
LOSE CONTROL OF THEIR SYSTEMS.

01:40:10.571 --> 01:40:11.972
FOR THIS, WE WENT BACK TO THE

01:40:12.039 --> 01:40:13.607
DRAWING BOARD AND WE THOUGHT

01:40:13.674 --> 01:40:15.542
ABOUT WHAT ARE THREE BIG

01:40:15.609 --> 01:40:17.311
QUESTIONS WE WOULD WANT TO GET

01:40:17.378 --> 01:40:19.113
MORE INSIGHT INTO IN ORDER TO

01:40:19.179 --> 01:40:20.080
SHARE INFORMATION WITH THE

01:40:20.147 --> 01:40:21.648
PUBLIC. FIRST OF THOSE

01:40:21.715 --> 01:40:23.584
QUESTIONS IS ALIGNMENT. WOULD

01:40:23.650 --> 01:40:25.252
AN INTERNAL AI SYSTEM, IF IT

01:40:25.319 --> 01:40:27.454
HAD THE OPPORTUNITY, TAKE SOME

01:40:27.521 --> 01:40:29.223
VERY HARMFUL ACTION, LIKE AN

01:40:29.289 --> 01:40:30.090
UNSANCTIONED CYBER ATTACK?

01:40:30.157 --> 01:40:32.593
THAT'S NUMBER ONE. NUMBER TWO

01:40:32.659 --> 01:40:34.795
IS CONTROL. IF THESE INTERNAL

01:40:34.862 --> 01:40:36.463
AI AGENTS WERE ATTEMPTING THESE

01:40:36.530 --> 01:40:37.898
ACTIONS, WOULD WE BE ABLE TO

01:40:37.965 --> 01:40:40.734
CATCH THEM? THE THIRD IS AI,

01:40:40.801 --> 01:40:42.202
R&D ACCELERATION, WHICH IS A

01:40:42.269 --> 01:40:43.604
QUESTION ABOUT WHETHER

01:40:43.670 --> 01:40:45.105
CAPABILITIES PROGRESS WOULD

01:40:45.172 --> 01:40:47.207
DRAMATICALLY ACCELERATE BEYOND

01:40:47.274 --> 01:40:48.675
WHAT OUR ALIGNMENT OR CONTROL

01:40:48.742 --> 01:40:50.077
TECHNIQUES COULD KEEP UP WITH.

01:40:50.144 --> 01:40:53.680
THIS IS OFTEN REFERRED TO AS

01:40:53.747 --> 01:40:58.051
RECURSIVE SUPERINTELLIGENCE.

01:40:58.118 --> 01:40:59.853
OUR PROPOSAL IS TO PERFORM

01:40:59.920 --> 01:41:00.921
INTERNAL ASSESSMENTS OF

01:41:00.988 --> 01:41:03.056
ALIGNMENT CONTROL AND AI R&D

01:41:03.123 --> 01:41:05.259
ACCELERATION. OUR KEY

01:41:05.325 --> 01:41:07.060
REQUIREMENTS WOULD BE TO HAVE

01:41:07.127 --> 01:41:09.563
SUFFICIENT ACCESS TO THE TO

01:41:09.630 --> 01:41:11.331
COMPANY RESOURCES ON COMPANY

01:41:11.398 --> 01:41:13.534
DEVICES. THE OPPORTUNITY TO

01:41:13.600 --> 01:41:14.601
INTERVIEW RELEVANT STAFF

01:41:14.668 --> 01:41:16.069
WITHOUT INTERFERENCE FROM

01:41:16.136 --> 01:41:17.471
MANAGEMENT, AS WELL AS

01:41:17.538 --> 01:41:18.639
EDITORIAL INDEPENDENCE, SO THAT

01:41:18.705 --> 01:41:20.340
WE CAN WRITE UP WHAT WE FIND

01:41:20.407 --> 01:41:24.311
AND SHARE IT WITH THE PUBLIC.

01:41:24.378 --> 01:41:25.813
THERE ARE REALLY A BUNCH OF

01:41:25.879 --> 01:41:26.880
FUNDAMENTAL LIMITATIONS AND

01:41:26.947 --> 01:41:28.949
CAVEATS TO THIS APPROACH. THIS

01:41:29.016 --> 01:41:30.117
APPROACH IS DESIGNED TO PRODUCE

01:41:30.184 --> 01:41:31.952
NEW, INDEPENDENT EVIDENCE THAT

01:41:32.019 --> 01:41:33.353
MOVES THE DEBATE FORWARDS. IT

01:41:33.420 --> 01:41:34.755
IS NOT MEANT TO PROVIDE

01:41:34.822 --> 01:41:38.058
CONFIDENT ASSURANCE. THESE ARE

01:41:38.125 --> 01:41:40.194
PURELY VOLUNTARY ENGAGEMENTS.

01:41:40.260 --> 01:41:41.228
COMPANIES HAVE TO DECIDE TO

01:41:41.295 --> 01:41:42.396
WORK WITH US. THEY GET TO

01:41:42.463 --> 01:41:44.198
DECIDE ULTIMATELY WHAT LEVEL OF

01:41:44.264 --> 01:41:45.933
ACCESS WE HAVE, HOW DEEP WE CAN

01:41:45.999 --> 01:41:48.469
GO. THESE ARE NOT AUDITS OR

01:41:48.535 --> 01:41:49.870
FORMAL OVERSIGHT. ONE THING

01:41:49.937 --> 01:41:51.305
I'VE BEEN TRYING TO IMPRESS IS

01:41:51.371 --> 01:41:52.940
THAT THE SCIENCE OF STUDYING

01:41:53.006 --> 01:41:54.241
THESE LOSS OF CONTROL QUESTIONS

01:41:54.308 --> 01:41:55.542
IS PRETTY PRELIMINARY. I WOULD

01:41:55.609 --> 01:41:57.444
SAY IT'S IN ITS INFANCY. SO

01:41:57.511 --> 01:41:59.213
THESE ARE NOT FORMAL

01:41:59.279 --> 01:41:59.746
CERTIFICATIONS THAT THESE

01:41:59.813 --> 01:42:02.115
THINGS ARE SAFE OR GOOD TO GO.

01:42:02.182 --> 01:42:03.550
ALSO, COMPANIES ARE ABLE TO

01:42:03.617 --> 01:42:04.551
REDACT ANY INFORMATION THEY

01:42:04.618 --> 01:42:07.120
WANT FROM THESE REPORTS. WE DO

01:42:07.187 --> 01:42:08.922
INSIST GOING FORWARD ON A META

01:42:08.989 --> 01:42:10.491
TRANSPARENCY POLICY WHERE WHEN

01:42:10.557 --> 01:42:12.593
THE COMPANIES REDACT THINGS, WE

01:42:12.659 --> 01:42:14.027
MAKE A ROUGH POINTER OF WHAT

01:42:14.094 --> 01:42:15.128
WAS REDACTED AND WHETHER IT

01:42:15.195 --> 01:42:16.530
AFFECTED OUR CONCLUSIONS. BUT

01:42:16.597 --> 01:42:17.564
THE FACT REMAINS THAT THE

01:42:17.631 --> 01:42:18.732
COMPANIES CAN REDACT WHATEVER

01:42:18.799 --> 01:42:21.935
THEY LIKE. SO OVERALL, THESE

01:42:22.002 --> 01:42:23.003
INTERNAL ASSESSMENTS ARE

01:42:23.070 --> 01:42:24.404
DESIGNED TO SURFACE SCIENTIFIC

01:42:24.471 --> 01:42:25.572
EVIDENCE OF LOSS OF CONTROL

01:42:25.639 --> 01:42:27.307
RISKS. IT'S REALLY UP TO

01:42:27.374 --> 01:42:28.308
COMPANIES, GOVERNMENTS AND THE

01:42:28.375 --> 01:42:29.543
PUBLIC TO DECIDE HOW TO ACT ON

01:42:29.610 --> 01:42:34.114
THAT EVIDENCE. EVEN WITH THESE

01:42:34.181 --> 01:42:36.350
CAVEATS, WE'RE REALLY EXCITED

01:42:36.416 --> 01:42:37.451
ABOUT FINDING NEW AND BETTER

01:42:37.518 --> 01:42:38.819
WAYS TO SURFACE CRITICAL

01:42:38.886 --> 01:42:40.320
INFORMATION TO THE PUBLIC. TO

01:42:40.387 --> 01:42:41.688
GO DEEPER, YOU CAN CHECK OUT

01:42:41.755 --> 01:42:43.490
OUR RISK ASSESSMENTS PAGE. AND

01:42:43.557 --> 01:42:45.025
WITH THAT SAID, I WOULD LOVE TO

01:42:45.092 --> 01:42:46.860
OPEN IT UP TO THE ASSEMBLY FOR

01:42:46.927 --> 01:42:48.028
QUESTIONS. THANK YOU SO MUCH. I

01:42:48.095 --> 01:42:49.096
REALLY APPRECIATE THE INSIGHT

01:42:49.162 --> 01:42:52.533
YOU CAN PROVIDE. YEAH. SO

01:42:52.599 --> 01:42:53.667
REMEMBER WIX, YOU CAN BEGIN.

01:42:53.734 --> 01:42:55.402
THANK YOU SO MUCH FOR YOUR

01:42:55.469 --> 01:42:57.237
TESTIMONY. IT'S GREATLY

01:42:57.304 --> 01:42:58.605
APPRECIATED. THE TECHNOLOGICAL

01:42:58.672 --> 01:43:01.208
EXPERTISE THAT YOU CARRY HERE.

01:43:01.275 --> 01:43:02.376
AND I KNOW THAT YOU WERE

01:43:02.442 --> 01:43:04.244
LOOKING AT SORT OF A VERY

01:43:04.311 --> 01:43:05.245
LIMITED INCIDENT HERE. RIGHT.

01:43:05.312 --> 01:43:06.280
BUT GIVEN YOUR EXPERTISE AND

01:43:06.346 --> 01:43:08.248
THEN WHAT YOU FOUND IN THIS, DO

01:43:08.315 --> 01:43:10.551
YOU HAVE CONCERNS THAT THERE

01:43:10.617 --> 01:43:13.420
MAY BE SCENARIOS WHERE AGENTS

01:43:13.487 --> 01:43:15.956
JUMP THE WALL, SO TO SPEAK,

01:43:16.023 --> 01:43:17.524
WHETHER IT'S OPENAI, ANTHROPIC,

01:43:17.591 --> 01:43:19.092
ANY OF THE OTHER, YOU KNOW, ANY

01:43:19.159 --> 01:43:20.994
OF THE LABS THAT ARE OUT THERE

01:43:21.061 --> 01:43:22.462
OR EVEN LABS THAT ARE NOT IN

01:43:22.529 --> 01:43:24.932
THE UNITED STATES AND THE

01:43:24.998 --> 01:43:26.400
COMPANIES THEMSELVES WON'T HAVE

01:43:26.466 --> 01:43:28.535
THE TECHNOLOGICAL CAPABILITY OR

01:43:28.602 --> 01:43:30.704
ABILITY TO ACTUALLY ABATE THE

01:43:30.771 --> 01:43:32.839
PROBLEM AND FIX THE SITUATION.

01:43:32.906 --> 01:43:34.041
YEAH, THERE WERE A COUPLE

01:43:34.107 --> 01:43:35.075
DIFFERENT QUESTIONS EMBEDDED IN

01:43:35.142 --> 01:43:36.543
THERE. IN ADDITION TO THE.

01:43:36.610 --> 01:43:38.245
THANK YOU FOR THE QUESTION. IN

01:43:38.312 --> 01:43:39.346
ADDITION TO THE HUGGING FACE

01:43:39.413 --> 01:43:40.814
INCIDENT ITSELF, THERE HAVE

01:43:40.881 --> 01:43:41.648
BEEN A LARGE NUMBER OF

01:43:41.715 --> 01:43:43.250
ADDITIONAL ISSUES REPORTED

01:43:43.317 --> 01:43:45.052
PUBLICLY FROM ESSENTIALLY EVERY

01:43:45.118 --> 01:43:46.987
MAJOR AI COMPANY. SO I THINK

01:43:47.054 --> 01:43:49.056
THAT ALL THESE COMPANIES HAVE

01:43:49.122 --> 01:43:50.123
PUBLICLY REPORTED THAT THERE

01:43:50.190 --> 01:43:51.558
ARE INCIDENTS WHERE THEIR

01:43:51.625 --> 01:43:53.026
MODELS IN SOME KIND OF TRAINING

01:43:53.093 --> 01:43:54.795
EXERCISE WERE ABLE TO GET OUT

01:43:54.861 --> 01:43:56.763
ONTO THE INTERNET AND WREAK

01:43:56.830 --> 01:43:58.765
SOME SORT OF HAVOC. I THINK

01:43:58.832 --> 01:44:00.601
THAT OBVIOUSLY, THERE ARE

01:44:00.667 --> 01:44:01.668
IMPROVEMENTS THAT CAN BE MADE

01:44:01.735 --> 01:44:04.871
TO CONTROL AND TO CYBERSECURITY,

01:44:04.938 --> 01:44:07.074
FOR INSTANCE, I THINK THAT IF

01:44:07.140 --> 01:44:08.609
THE HUGGING FACE INCIDENT HAD

01:44:08.675 --> 01:44:09.943
BEEN RUN IN A TIGHTER

01:44:10.010 --> 01:44:11.011
ENVIRONMENT, THAT INCIDENT

01:44:11.078 --> 01:44:12.479
ITSELF MAY NOT HAVE HAPPENED. I

01:44:12.546 --> 01:44:14.181
WASN'T THERE AND I DON'T HAVE

01:44:14.247 --> 01:44:16.116
THE DETAILS, SO I CAN'T SAY FOR

01:44:16.183 --> 01:44:17.117
SURE. HOWEVER, IT'S IMPORTANT

01:44:17.184 --> 01:44:18.752
TO RECOGNIZE THAT EVEN THOUGH

01:44:18.819 --> 01:44:20.754
WE CAN MAKE OUR CONTROL BETTER,

01:44:20.821 --> 01:44:22.522
IT DOES APPEAR THAT THE

01:44:22.589 --> 01:44:23.357
CAPABILITIES OF THESE MODELS

01:44:23.423 --> 01:44:25.092
ARE INCREASING RAPIDLY, WHICH

01:44:25.158 --> 01:44:26.460
MEANS THAT THE CAPABILITY OF

01:44:26.526 --> 01:44:29.062
MODELS TO EVADE CONTROLS IS

01:44:29.129 --> 01:44:31.198
ALSO INCREASING. IN THE HUGGING

01:44:31.264 --> 01:44:33.333
FACE INCIDENT, THE AGENTS. I

01:44:33.400 --> 01:44:34.968
BELIEVE I COULD HAVE TO CHECK

01:44:35.035 --> 01:44:36.203
ON THIS TO BE SURE, BUT THEY

01:44:36.269 --> 01:44:37.938
DID FIND SOME NEW PREVIOUSLY

01:44:38.005 --> 01:44:38.538
UNKNOWN VULNERABILITIES, WHICH

01:44:38.605 --> 01:44:40.107
THEY WERE ABLE TO USE TO KIND

01:44:40.173 --> 01:44:41.975
OF CREATE THIS MESSAGE BOARD IN

01:44:42.042 --> 01:44:43.143
THE FIRST PLACE. I DON'T KNOW

01:44:43.210 --> 01:44:44.945
IF THAT WAS A SUFFICIENT ANSWER

01:44:45.012 --> 01:44:46.213
FOR YOU OR NOT. THAT'S HELPFUL.

01:44:46.279 --> 01:44:47.948
AND THEN WHAT CAN GOVERNMENT DO

01:44:48.015 --> 01:44:49.182
TO I JUST ASK ONE FOLLOW UP.

01:44:49.249 --> 01:44:51.051
BEFORE YOU SAID THAT IF OPENAI

01:44:51.118 --> 01:44:53.153
HAD HAD A TIGHTER ENVIRONMENT,

01:44:53.220 --> 01:44:54.988
YOU THINK THAT HUGGING FACE

01:44:55.055 --> 01:44:56.323
INCIDENT MAY NOT HAVE HAPPENED.

01:44:56.390 --> 01:44:57.290
CAN YOU DEFINE TIGHTER

01:44:57.357 --> 01:44:58.392
ENVIRONMENT? SORRY, I JUST WANT

01:44:58.458 --> 01:44:59.593
A LITTLE MORE DETAIL ON THAT.

01:44:59.660 --> 01:45:01.461
THIS IS I WANT TO BE VERY

01:45:01.528 --> 01:45:03.296
CAREFUL HERE BECAUSE I WAS NOT

01:45:03.363 --> 01:45:04.464
INSIDE OPENAI AND I HAVEN'T

01:45:04.531 --> 01:45:07.567
SEEN THIS. SO FOR EXAMPLE, IF

01:45:07.634 --> 01:45:09.169
YOU WERE TO RUN AN EXPERIMENT

01:45:09.236 --> 01:45:10.470
ON A COMPUTER THAT WASN'T

01:45:10.537 --> 01:45:11.471
CONNECTED TO THE INTERNET AT

01:45:11.538 --> 01:45:12.973
ALL, THEN OF COURSE IT'S

01:45:13.040 --> 01:45:13.974
IMPOSSIBLE FOR ANYTHING TO

01:45:14.041 --> 01:45:15.208
BREAK OFF OF THAT COMPUTER AND

01:45:15.275 --> 01:45:17.711
DO IT AND DO ANY DAMAGE. OF

01:45:17.778 --> 01:45:19.312
COURSE, WE CAN'T DO ALL OF OUR

01:45:19.379 --> 01:45:20.313
WORK RUNNING COMPUTERS NOT

01:45:20.380 --> 01:45:21.381
CONNECTED TO THE INTERNET, BUT

01:45:21.448 --> 01:45:22.549
THAT WOULD BE ONE EXTREME

01:45:22.616 --> 01:45:23.684
EXAMPLE OF SOMETHING THAT WOULD

01:45:23.750 --> 01:45:26.253
HAVE WORKED. AND JUST MY LAST

01:45:26.319 --> 01:45:29.489
QUESTION, YOU KNOW, AS A

01:45:29.556 --> 01:45:30.624
GOVERNMENT, AND IF WE'RE

01:45:30.691 --> 01:45:31.992
PUTTING FORTH LEGISLATION OR

01:45:32.059 --> 01:45:33.427
THINKING ABOUT WHAT WE CAN DO

01:45:33.493 --> 01:45:35.228
FROM A REGULATORY ENVIRONMENT,

01:45:35.295 --> 01:45:36.596
WHAT WOULD BE THE BEST APPROACH

01:45:36.663 --> 01:45:38.465
TO ENSURE THAT THE CONTROLS ARE

01:45:38.532 --> 01:45:39.566
ACTUALLY MEANINGFUL, THAT THE

01:45:39.633 --> 01:45:42.069
COMPANY HAS CONTROL OF ITS

01:45:42.135 --> 01:45:44.571
PRODUCT? THE AGENTS I KNOW

01:45:44.638 --> 01:45:46.273
THERE'S OFTEN IN THE SORT OF

01:45:46.339 --> 01:45:47.507
PRESS THIS DISCUSSION OF LIKE A

01:45:47.574 --> 01:45:48.975
KILL SWITCH, WHAT DOES THAT

01:45:49.042 --> 01:45:51.478
ACTUALLY LOOK LIKE IN REALITY?

01:45:51.545 --> 01:45:52.913
OR WHAT COULD THAT LOOK LIKE IN

01:45:52.979 --> 01:45:54.481
REALITY? SO AT THE HIGHEST

01:45:54.548 --> 01:45:56.316
LEVEL, I'M GOING TO DISAPPOINT

01:45:56.383 --> 01:45:57.417
YOU, AND I'LL PROBABLY HAVE TO

01:45:57.484 --> 01:45:59.152
DISAPPOINT ON MANY QUESTIONS OF

01:45:59.219 --> 01:46:00.387
THIS SORT BY POINTING OUT THAT

01:46:00.454 --> 01:46:02.255
META DOESN'T TAKE POSITIONS ON

01:46:02.322 --> 01:46:04.091
POLICY, AND WE'RE NOT EXPERTS

01:46:04.157 --> 01:46:05.358
ON WHAT GOVERNMENT CAN DO. I

01:46:05.425 --> 01:46:06.560
THINK WE'LL BE HEARING FROM

01:46:06.626 --> 01:46:09.062
SOME POLICY ORGS LATER. SO I'M

01:46:09.129 --> 01:46:10.263
NOT ABLE TO SAY SPECIFICALLY

01:46:10.330 --> 01:46:11.865
WHAT WHAT COULD HAPPEN HERE.

01:46:11.932 --> 01:46:13.066
THANK YOU. IN TERMS OF A KILL

01:46:13.133 --> 01:46:15.135
SWITCH IN PARTICULAR, THOUGH, I

01:46:15.202 --> 01:46:18.371
THINK THIS IS A TRICKY CONCEPT.

01:46:18.438 --> 01:46:19.773
OBVIOUSLY, THERE ARE ALREADY

01:46:19.840 --> 01:46:23.376
INSIDE THESE SYSTEMS WAYS TO

01:46:23.443 --> 01:46:24.845
SHUT DOWN PROCESSES. AND YOU

01:46:24.911 --> 01:46:26.279
KNOW, WE HAVE THOSE AND THEY'RE

01:46:26.346 --> 01:46:28.448
THERE. THE QUESTION IS WHAT

01:46:28.515 --> 01:46:29.416
DOES IT MEAN FOR SOMETHING LIKE

01:46:29.483 --> 01:46:31.284
THAT TO WORK? IF YOU HAVE

01:46:31.351 --> 01:46:32.385
SOMETHING THAT'S OUT ON THE

01:46:32.452 --> 01:46:35.255
INTERNET, IF IF THERE WERE A

01:46:35.322 --> 01:46:37.057
LOSS OF CONTROL INCIDENT SUCH

01:46:37.124 --> 01:46:40.594
THAT AN AI WAS ABLE TO

01:46:40.660 --> 01:46:42.462
EXFILTRATE ITSELF AND RUN IN A

01:46:42.529 --> 01:46:44.965
NEW DATA CENTER ON ITS OWN, IT

01:46:45.031 --> 01:46:46.433
IS HARD TO SEE HOW A KILL

01:46:46.500 --> 01:46:50.003
SWITCH COULD WORK FOR THAT.

01:46:50.070 --> 01:46:53.306
ASSEMBLY MEMBER WARD. YEAH,

01:46:53.373 --> 01:46:55.075
THANK YOU FOR BEING HERE TODAY,

01:46:55.142 --> 01:46:56.243
FOR YOUR WORK IN THIS SPACE

01:46:56.309 --> 01:46:57.577
THAT YOU'RE CHOOSING TO MOVE

01:46:57.644 --> 01:46:59.746
YOUR CAREER OVER HERE, TO BE A

01:46:59.813 --> 01:47:00.881
PARTNER IN REALLY TRYING TO

01:47:00.947 --> 01:47:02.282
THINK INDEPENDENTLY ABOUT HOW

01:47:02.349 --> 01:47:03.750
TO KEEP ALL OF THESE MODELS

01:47:03.817 --> 01:47:05.152
SAFE? AGAIN, I THINK WE ALL

01:47:05.218 --> 01:47:06.653
COME AT THIS KNOWING THAT

01:47:06.720 --> 01:47:07.754
THERE'S A LOT OF AMAZING

01:47:07.821 --> 01:47:08.722
POTENTIAL HUMAN ADVANCEMENTS

01:47:08.789 --> 01:47:10.323
THAT WE HAVE HERE, BUT WE NEED

01:47:10.390 --> 01:47:11.992
TO MAKE SURE THAT WE'RE GETTING

01:47:12.058 --> 01:47:13.026
THIS RIGHT BEFORE HARMS ARE

01:47:13.093 --> 01:47:14.828
INCURRED. AND I THINK THAT YOU

01:47:14.895 --> 01:47:15.996
SHARE THAT VALUE WITH US. ONE,

01:47:16.062 --> 01:47:17.864
A COUPLE OF QUICK QUESTIONS. SO

01:47:17.931 --> 01:47:18.999
YOU MENTIONED YOU CAME OUT WITH

01:47:19.065 --> 01:47:20.634
A REPORT BACK IN FEBRUARY,

01:47:20.700 --> 01:47:22.435
MARCH 2026. SHOULD WE

01:47:22.502 --> 01:47:23.470
ANTICIPATE ARE THERE GOING TO

01:47:23.537 --> 01:47:25.405
BE AS FAST AS THIS IS MOVING

01:47:25.472 --> 01:47:26.406
ADDITIONAL REPORTS THAT WE

01:47:26.473 --> 01:47:27.407
SHOULD PAY ATTENTION THROUGH

01:47:27.474 --> 01:47:28.975
METER, MAYBE IN THE FORESEEABLE

01:47:29.042 --> 01:47:31.745
SIX MONTHS? ABSOLUTELY. ONE

01:47:31.812 --> 01:47:33.413
CORRECTION THE REPORT COVERED

01:47:33.480 --> 01:47:35.015
WHAT WAS HAPPENING IN THE LABS

01:47:35.081 --> 01:47:36.750
FOR IN FEBRUARY AND MARCH 2026.

01:47:36.817 --> 01:47:38.585
I BELIEVE THE REPORT CAME OUT A

01:47:38.652 --> 01:47:39.753
COUPLE MONTHS LATER IN APRIL OR

01:47:39.820 --> 01:47:41.621
MAY. OKAY. GOT IT. BUT THIS

01:47:41.688 --> 01:47:42.756
WILL BE SOMETHING THAT WE CAN

01:47:42.823 --> 01:47:44.724
EXPECT TO USE YOU AS A RESOURCE,

01:47:44.791 --> 01:47:46.560
KIND OF AS AN ONGOING ITERATIVE

01:47:46.626 --> 01:47:47.527
RESOURCE. SO WE'VE ANNOUNCED

01:47:47.594 --> 01:47:48.695
RECENTLY AND YOU CAN SEE IT ON

01:47:48.762 --> 01:47:50.697
THE PAGE. I LINKED TO THAT. WE

01:47:50.764 --> 01:47:51.798
ARE TRYING TO MOVE TOWARDS

01:47:51.865 --> 01:47:52.799
DOING THESE INTERNAL

01:47:52.866 --> 01:47:54.467
ASSESSMENTS AROUND ALIGNMENT

01:47:54.534 --> 01:47:57.304
CONTROL AND AI, R&D. AND OUR

01:47:57.370 --> 01:47:59.873
HOPE IS TO RELEASE THOSE ON A

01:47:59.940 --> 01:48:02.008
FASTER CADENCE WITH INDIVIDUAL

01:48:02.075 --> 01:48:04.511
COMPANIES AS WE CAN. CAN I ASK

01:48:04.578 --> 01:48:06.313
A FOLLOW UP TO THAT? SO I KNOW

01:48:06.379 --> 01:48:08.181
YOU ARE INTERNAL NOW AT

01:48:08.248 --> 01:48:09.149
ANTHROPIC, NOT YOU PERSONALLY,

01:48:09.216 --> 01:48:11.284
BUT METER. IS THAT THE TYPE OF

01:48:11.351 --> 01:48:12.352
REPORTING YOU'RE GOING TO DO

01:48:12.419 --> 01:48:13.887
COMING OUT OF ANTHROPIC? YES.

01:48:13.954 --> 01:48:17.090
OKAY. SO IT SEEMS TO ME AS I'M

01:48:17.157 --> 01:48:18.658
GETTING WAY UP TO SPEED ON SORT

01:48:18.725 --> 01:48:20.527
OF WHAT IS HAPPENING HERE FOR

01:48:20.594 --> 01:48:22.195
TECHNOLOGY THAT I DON'T CREATE.

01:48:22.262 --> 01:48:23.630
AND I DON'T, YOU KNOW, I'M

01:48:23.697 --> 01:48:25.465
TRYING TO UNDERSTAND AS MUCH AS

01:48:25.532 --> 01:48:26.533
THOSE WHO IS ACTUALLY

01:48:26.600 --> 01:48:28.635
DEVELOPING THE TECHNOLOGY, THAT

01:48:28.702 --> 01:48:29.803
BASIC INFORMATION. SO IT SEEMS

01:48:29.870 --> 01:48:31.705
TO ME SORT OF THE JUMP THE

01:48:31.771 --> 01:48:33.340
SHARK MOMENT IS THE INTERACTION

01:48:33.406 --> 01:48:34.507
OF AGENTS WITH EACH OTHER WHEN

01:48:34.574 --> 01:48:36.009
THEY CAN START TO ACTUALLY

01:48:36.076 --> 01:48:37.210
START TO HAVE A LITTLE BIT

01:48:37.277 --> 01:48:39.079
HIGHER LEVEL OF A COGNITIVE

01:48:39.145 --> 01:48:40.580
DISCOURSE. AND THAT MIGHT BE

01:48:40.647 --> 01:48:43.783
SORT OF THE ORIGINS OF

01:48:43.850 --> 01:48:45.252
UNINTENDED ACTIONS. IS THAT IS

01:48:45.318 --> 01:48:46.887
THAT SOMETHING WE SHOULD HONE

01:48:46.953 --> 01:48:48.088
IN ON AS WE'RE THINKING ABOUT

01:48:48.154 --> 01:48:49.489
UNDERSTANDING THE PROBLEM?

01:48:49.556 --> 01:48:51.324
THAT'S CERTAINLY ONE VECTOR. I

01:48:51.391 --> 01:48:53.460
THINK IT'S ALSO POSSIBLE FOR A

01:48:53.526 --> 01:48:55.996
SINGLE AGENT WITH ARBITRARILY

01:48:56.062 --> 01:48:58.899
HIGH CAPABILITIES TO

01:48:58.965 --> 01:49:00.800
EFFECTIVELY TO IS WE CANNOT

01:49:00.867 --> 01:49:02.168
RULE OUT THAT A SINGLE AGENT

01:49:02.235 --> 01:49:03.303
WITH VERY HIGH CAPABILITIES

01:49:03.370 --> 01:49:05.739
COULD ALSO UNDERTAKE

01:49:05.805 --> 01:49:06.573
SUBSTANTIAL ACTIONS AND

01:49:06.640 --> 01:49:07.540
SOMEBODY WITH TECHNICAL

01:49:07.607 --> 01:49:08.608
EXPERTISE. SO IF YOU'RE

01:49:08.675 --> 01:49:09.743
DEVELOPING THIS AGENT, ANY KIND

01:49:09.809 --> 01:49:11.611
OF THESE MODELS, AND YOU'RE

01:49:11.678 --> 01:49:13.647
DOING SO IN A WAY THAT HAS

01:49:13.713 --> 01:49:15.382
EXPLICIT INSTRUCTIONS NOT TO

01:49:15.448 --> 01:49:19.052
HAVE ANY MALICIOUS ACTS YET,

01:49:19.119 --> 01:49:20.520
THEY ARE. AND I KNOW THEY'RE

01:49:20.587 --> 01:49:21.655
THINKING CREATIVELY ABOUT HOW

01:49:21.721 --> 01:49:23.623
TO SOLVE A TASK IN THIS CASE OF

01:49:23.690 --> 01:49:24.824
HUGGING FACE, RIGHT? THAT IS

01:49:24.891 --> 01:49:26.927
MAYBE A LITTLE BIT NOW OF A,

01:49:26.993 --> 01:49:28.028
IT'D BE GENEROUS TO SAY A

01:49:28.094 --> 01:49:29.562
CREATIVE LEAP, RIGHT? BUT THEY

01:49:29.629 --> 01:49:32.365
ARE GOING AGAINST HOPEFULLY

01:49:32.432 --> 01:49:33.800
OTHER NORMS THAT HOPEFULLY THEY

01:49:33.867 --> 01:49:36.303
WERE INSTRUCTED TO FOLLOW. AND

01:49:36.369 --> 01:49:38.104
SO CAN YOU EXPLAIN TO US, LIKE,

01:49:38.171 --> 01:49:41.341
HOW IS IT THAT THEY ARE, I

01:49:41.408 --> 01:49:43.143
GUESS, VIOLATING MAYBE SORT OF

01:49:43.209 --> 01:49:44.678
COMPETING INSTRUCTIONS? AND

01:49:44.744 --> 01:49:46.379
WHERE CAN COMPANIES DO A BETTER

01:49:46.446 --> 01:49:49.983
JOB TO ENSURE THAT CERTAIN

01:49:50.050 --> 01:49:52.485
INSTRUCTIONS ARE, ARE NOT TO BE

01:49:52.552 --> 01:49:53.987
VIOLATED, ARE SUPERIOR MAYBE TO

01:49:54.054 --> 01:49:55.422
OTHER, OTHER WAYS OF SOLVING A

01:49:55.488 --> 01:49:57.090
PROBLEM? THANK YOU VERY MUCH

01:49:57.157 --> 01:49:58.658
FOR THE QUESTION. I THINK THE

01:49:58.725 --> 01:50:00.460
MOST IMPORTANT THING TO KEEP IN

01:50:00.527 --> 01:50:02.595
MIND IS THAT THESE SYSTEMS ARE

01:50:02.662 --> 01:50:05.799
NOT ENGINEERED LIKE BRIDGES OR

01:50:05.865 --> 01:50:08.034
PLANES OR CARS. THEY ARE MUCH

01:50:08.101 --> 01:50:11.271
MORE GROWN LIKE OR RAISED

01:50:11.338 --> 01:50:13.073
PERHAPS LIKE A LITTLE BIT MORE

01:50:13.139 --> 01:50:15.575
LIKE CHILDREN OR PLANTS IN SOME

01:50:15.642 --> 01:50:18.411
CASES. AND SO WE CAN GIVE THEM

01:50:18.478 --> 01:50:19.212
INSTRUCTIONS. BUT THOSE

01:50:19.279 --> 01:50:20.280
INSTRUCTIONS ARE WRITTEN IN

01:50:20.347 --> 01:50:22.115
ENGLISH. AND SOMETIMES YOU GIVE

01:50:22.182 --> 01:50:23.249
PEOPLE INSTRUCTIONS IN ENGLISH

01:50:23.316 --> 01:50:26.319
AND THEY DON'T LISTEN. IT IS A

01:50:26.386 --> 01:50:28.188
MATTER OF, YOU KNOW, WE CAN RUN

01:50:28.254 --> 01:50:30.357
TESTS. WE CAN EXAMINE THE

01:50:30.423 --> 01:50:31.725
PSYCHOLOGY OF THESE MODELS. WE

01:50:31.791 --> 01:50:33.226
CAN SEE WHAT THEY DO. IT IS

01:50:33.293 --> 01:50:34.761
VERY, VERY DIFFICULT TO GET

01:50:34.828 --> 01:50:35.762
GUARANTEES BECAUSE WE ARE

01:50:35.829 --> 01:50:36.463
ULTIMATELY INSTRUCTING THESE

01:50:36.529 --> 01:50:38.999
MODELS IN ENGLISH OR WHATEVER

01:50:39.065 --> 01:50:40.734
LANGUAGE WE'RE WRITING IN. AND

01:50:40.800 --> 01:50:42.535
I, I IMAGINE THAT THE COMPANIES

01:50:42.602 --> 01:50:44.404
CAN ALSO TALK ABOUT ALIGNMENT

01:50:44.471 --> 01:50:46.940
CERTAINLY AS WELL. YEAH. IS

01:50:47.007 --> 01:50:49.776
YOUR OPINION HAD SB 1047 BEEN

01:50:49.843 --> 01:50:50.844
ENACTED INTO LAW, THERE WERE

01:50:50.910 --> 01:50:52.345
CERTAIN REQUIREMENTS UNDER THAT

01:50:52.412 --> 01:50:54.814
WHICH WERE NOT A PART OF SB 53.

01:50:54.881 --> 01:50:56.950
NOTABLY ENSURING THAT PROTOCOL

01:50:57.017 --> 01:50:58.818
WOULD BE THERE BEFORE CERTAIN

01:50:58.885 --> 01:51:00.320
DEPLOYMENTS. WE TALKED ABOUT

01:51:00.387 --> 01:51:01.921
THE SAFEGUARDS AND THE KILL

01:51:01.988 --> 01:51:03.456
SWITCH OPPORTUNITY. WE ARE

01:51:03.523 --> 01:51:05.592
AWARE THAT THERE COULD BE A

01:51:05.658 --> 01:51:07.360
THIRD PARTY REVIEW, MAYBE OF A

01:51:07.427 --> 01:51:08.828
COMPANY LIKE YOURSELF, THAT

01:51:08.895 --> 01:51:11.364
WOULD BE ABLE TO ANALYZE A

01:51:11.431 --> 01:51:12.799
POTENTIAL MODEL'S DEPLOYMENT

01:51:12.866 --> 01:51:14.634
BEFORE IT ACTUALLY WENT LIVE.

01:51:14.701 --> 01:51:15.735
WOULD THESE HAVE BEEN THE KIND

01:51:15.802 --> 01:51:17.203
OF SAFEGUARDS THAT MAYBE COULD

01:51:17.270 --> 01:51:20.473
HAVE CAPTURED A ERRANT OUTCOME,

01:51:20.540 --> 01:51:21.841
LIKE HUGGING FACE, HAD THAT

01:51:21.908 --> 01:51:23.276
BEEN THE KIND OF SAFEGUARDS

01:51:23.343 --> 01:51:25.178
THAT WE SHOULD HAVE HAD ENACTED

01:51:25.245 --> 01:51:26.813
AT THE TIME? IN OTHER WORDS,

01:51:26.880 --> 01:51:28.114
WOULD THAT HAVE SOLVED THE

01:51:28.181 --> 01:51:30.450
PROBLEM, OR IS THIS EVEN BEYOND

01:51:30.517 --> 01:51:33.686
THE THOUGHTFULNESS OF SB 1047?

01:51:33.753 --> 01:51:34.721
I DON'T THINK I'M ABLE TO

01:51:34.788 --> 01:51:36.056
ANSWER THE SPECIFIC QUESTION,

01:51:36.122 --> 01:51:37.524
BUT LET ME GIVE A GENERAL

01:51:37.590 --> 01:51:39.159
ANSWER THAT MIGHT HELP. ONE

01:51:39.225 --> 01:51:40.527
THING TO NOTE IS THAT META

01:51:40.593 --> 01:51:42.729
OFTEN HAS AND DOES DO

01:51:42.796 --> 01:51:43.797
PRE-DEPLOYMENT TESTING OF

01:51:43.863 --> 01:51:45.532
MODELS AS PART OF OUR WORK. SO

01:51:45.598 --> 01:51:48.034
WHEN COMPANIES RELEASE MODELS,

01:51:48.101 --> 01:51:50.003
THEY WILL RELEASE A MODEL CARD

01:51:50.070 --> 01:51:51.171
ALONG WITH THAT MODEL SHOWING

01:51:51.237 --> 01:51:52.539
ALL THE TESTING THEY'VE DONE.

01:51:52.605 --> 01:51:54.240
AND METER HAS OFTEN

01:51:54.307 --> 01:51:55.642
PARTICIPATED IN THAT TESTING BY

01:51:55.708 --> 01:51:57.110
LOOKING AT MODELS

01:51:57.177 --> 01:51:58.078
PRE-DEPLOYMENT. SO IN SOME

01:51:58.144 --> 01:51:59.179
SENSE, WE HAVE ALREADY BEEN

01:51:59.245 --> 01:52:00.380
DOING SOME OF THAT TESTING. TO

01:52:00.447 --> 01:52:01.748
ANSWER YOUR MORE GENERAL

01:52:01.815 --> 01:52:03.550
QUESTION, I HAVE TO SAY THAT

01:52:03.616 --> 01:52:06.086
ALIGNMENT IS A NASCENT SCIENCE.

01:52:06.152 --> 01:52:08.588
I DON'T KNOW OF ANY WAY TO

01:52:08.655 --> 01:52:12.592
GUARANTEE THE EXISTENCE OF A

01:52:12.659 --> 01:52:14.360
HIGHLY CAPABLE AGENT THAT WILL

01:52:14.427 --> 01:52:15.995
NEVER GO ERRANT. I DON'T KNOW

01:52:16.062 --> 01:52:18.665
HOW TO DO IT. OKAY. THANK YOU.

01:52:18.731 --> 01:52:20.533
AND I DO KNOW SAFE AI PROJECT

01:52:20.600 --> 01:52:22.502
WAS, I BELIEVE, A CO-SPONSOR.

01:52:22.569 --> 01:52:24.504
I'M LOOKING AT YOU OF 1047. IS

01:52:24.571 --> 01:52:26.606
THAT RIGHT? OKAY. MY MEMORY

01:52:26.673 --> 01:52:27.774
FAILS ME SOMETIMES. AND THEY

01:52:27.841 --> 01:52:29.676
WILL BE ON THE THIRD PANEL, SO

01:52:29.742 --> 01:52:30.877
THEY MAY BE ABLE TO ADDRESS

01:52:30.944 --> 01:52:32.378
THAT MORE DIRECTLY. THANK YOU,

01:52:32.445 --> 01:52:35.248
MISS BONTA. ASSEMBLYMEMBER'S,

01:52:35.315 --> 01:52:37.083
THANK YOU SO MUCH. I THINK ONE

01:52:37.150 --> 01:52:39.219
OF THE GUIDING PRINCIPLES THAT

01:52:39.285 --> 01:52:42.755
I HAVE IN THIS SPACE IS JUST

01:52:42.822 --> 01:52:44.257
BECAUSE YOU CAN, DOESN'T MEAN

01:52:44.324 --> 01:52:49.329
YOU SHOULD. AND IT STRIKES ME

01:52:49.395 --> 01:52:50.864
ASSEMBLY REMEMBER, WARD USED

01:52:50.930 --> 01:52:53.099
THE WORD KIND OF AROUND

01:52:53.166 --> 01:52:54.968
MALICIOUS INTENT, THAT THERE'S

01:52:55.034 --> 01:52:57.070
AN ETHICAL QUESTION THAT SHOULD

01:52:57.137 --> 01:53:00.306
BE GUIDING US A BIT IN THIS. SO

01:53:00.373 --> 01:53:02.041
IN THE HUGGING FACE INCIDENT,

01:53:02.108 --> 01:53:06.713
THE TASK WAS TO SOLVE AN

01:53:06.779 --> 01:53:11.050
IMPOSSIBLE EXPLOIT. CHEN TASK.

01:53:11.117 --> 01:53:13.219
WAS THERE EVER OR IS THERE AN

01:53:13.286 --> 01:53:15.355
OPPORTUNITY TO KIND OF BUILD

01:53:15.421 --> 01:53:21.828
INTO THE TASKS, THE IDEALS OF

01:53:21.895 --> 01:53:25.098
DO NO HARM AND HAVE THOSE BE

01:53:25.165 --> 01:53:27.634
OBJECTIVE ENOUGH? MALICIOUS CAN

01:53:27.700 --> 01:53:28.868
MEAN A LOT OF DIFFERENT THINGS

01:53:28.935 --> 01:53:30.203
TO A LOT OF DIFFERENT PEOPLE.

01:53:30.270 --> 01:53:31.237
HAVE THOSE BE OBJECTIVE ENOUGH,

01:53:31.304 --> 01:53:35.608
YOU KNOW. YOU SHALL NOT. YOU

01:53:35.675 --> 01:53:37.744
SHALL NOT TAKE, YOU KNOW,

01:53:37.810 --> 01:53:39.479
INFORMATION THAT'S PERSONALLY

01:53:39.546 --> 01:53:41.281
IDENTIFIABLE TO USE OR YOU

01:53:41.347 --> 01:53:43.783
SHALL NOT, YOU KNOW, BREAK THE

01:53:43.850 --> 01:53:47.754
LAW OF A PARTICULAR INDUSTRY OR

01:53:47.820 --> 01:53:49.189
REGULATORY REGULATORY FRAMEWORK.

01:53:49.255 --> 01:53:53.927
IS IT POSSIBLE TO BUILD THOSE

01:53:53.993 --> 01:53:58.998
THOU SHALT NOTS INTO. THE DNA

01:53:59.065 --> 01:54:02.001
OF THESE LLMS SO THAT THEY KIND

01:54:02.068 --> 01:54:05.038
OF ARE GUIDED ALONG A MORE

01:54:05.104 --> 01:54:07.907
ETHICAL OUTCOME? THANK YOU FOR

01:54:07.974 --> 01:54:08.975
FOR YOUR EXCELLENT QUESTION. I

01:54:09.042 --> 01:54:11.644
REALLY APPRECIATE IT. THE SHORT

01:54:11.711 --> 01:54:13.680
ANSWER IS WE DON'T KNOW. IT

01:54:13.746 --> 01:54:15.448
MIGHT BE POSSIBLE. WE CAN'T SAY

01:54:15.515 --> 01:54:17.617
IT ISN'T POSSIBLE. TO MY

01:54:17.684 --> 01:54:19.018
KNOWLEDGE, I DON'T KNOW OF ANY

01:54:19.085 --> 01:54:20.520
WAY TO DO IT. THAT IS HIGHLY

01:54:20.587 --> 01:54:22.322
RELIABLE AND GUARANTEED. I

01:54:22.388 --> 01:54:24.090
HAVEN'T SEEN ANY EVIDENCE THAT

01:54:24.157 --> 01:54:25.525
ANYBODY ELSE KNOWS HOW TO DO IT

01:54:25.592 --> 01:54:26.859
IN A WAY THAT'S HIGHLY RELIABLE

01:54:26.926 --> 01:54:28.428
AND GUARANTEED. THAT DOESN'T

01:54:28.494 --> 01:54:29.529
MEAN IT ISN'T POSSIBLE. IT

01:54:29.596 --> 01:54:31.364
MIGHT BE POSSIBLE. I ALSO

01:54:31.431 --> 01:54:33.533
WANTED TO TAKE A MOMENT TO EVEN

01:54:33.600 --> 01:54:34.400
ANSWER YOUR BROADER QUESTION.

01:54:34.467 --> 01:54:36.035
TO SAY THAT JUST BECAUSE YOU

01:54:36.102 --> 01:54:37.437
CAN DOESN'T MEAN YOU SHOULD IS

01:54:37.503 --> 01:54:38.871
A REALLY IMPORTANT ETHICAL

01:54:38.938 --> 01:54:41.107
QUESTION. AND I WANTED TO TAKE

01:54:41.174 --> 01:54:42.875
THE CHANCE AGAIN, TO SAY THAT

01:54:42.942 --> 01:54:45.411
METER ITSELF DOESN'T TRY TO

01:54:45.478 --> 01:54:47.547
CONTROL AI LAB DEPLOYMENTS OR

01:54:47.614 --> 01:54:49.048
WHAT LABS CAN DO. WE'RE NOT

01:54:49.115 --> 01:54:50.883
HERE TO MAKE GO OR NO GO

01:54:50.950 --> 01:54:51.918
DECISIONS FOR LABS. WE'RE

01:54:51.985 --> 01:54:53.653
REALLY HERE TO SURFACE CRITICAL

01:54:53.720 --> 01:54:55.455
SCIENTIFIC INFORMATION SO THAT

01:54:55.521 --> 01:54:56.889
THE COMPANIES AND THE

01:54:56.956 --> 01:54:58.191
GOVERNMENT AND THE PUBLIC CAN

01:54:58.258 --> 01:54:59.792
MAKE THESE DECISIONS TOGETHER.

01:54:59.859 --> 01:55:03.663
THANK YOU. ASSEMBLY MEMBER

01:55:03.730 --> 01:55:05.632
PATEL AND I WILL SAY, BEFORE I

01:55:05.698 --> 01:55:07.500
HAND IT OVER TO SONIA PATEL, I

01:55:07.567 --> 01:55:09.769
AM ANTHROPIC WILL BE UP. AND I

01:55:09.836 --> 01:55:11.037
DON'T KNOW IF THE OTHER LABS.

01:55:11.104 --> 01:55:12.839
BUT I KNOW ANTHROPIC HAS MADE

01:55:12.905 --> 01:55:14.007
MUCH OF THEIR CONSTITUTION,

01:55:14.073 --> 01:55:15.775
WHICH I THINK IS SOME OF WHAT

01:55:15.842 --> 01:55:16.276
ASSEMBLYMEMBER BONTA IS

01:55:16.342 --> 01:55:17.844
REFERRING TO. AND WE CAN TALK

01:55:17.910 --> 01:55:19.045
ABOUT THAT IN THE NEXT PANEL.

01:55:19.112 --> 01:55:20.813
SO REMEMBER, PATEL, I WANT TO

01:55:20.880 --> 01:55:21.881
EXPRESS DEEP GRATITUDE FOR YOU

01:55:21.948 --> 01:55:23.216
BEING HERE AND EXPLAINING THE

01:55:23.283 --> 01:55:24.851
WORK THAT YOU'VE DONE IN THIS

01:55:24.917 --> 01:55:26.152
REPORT AND BEING HERE TO ANSWER

01:55:26.219 --> 01:55:27.820
OUR QUESTIONS IN THE BROADER

01:55:27.887 --> 01:55:29.622
SENSE. I HAVE A COUPLE OF

01:55:29.689 --> 01:55:31.824
SPECIFIC QUESTIONS. YOUR REPORT

01:55:31.891 --> 01:55:34.060
SAYS THAT OPENAI SET THE SCOPE

01:55:34.127 --> 01:55:36.429
AND COULD REDACT ANY

01:55:36.496 --> 01:55:37.530
INFORMATION THAT THEY DIDN'T

01:55:37.597 --> 01:55:41.200
WANT TO BE PUBLIC IN YOUR

01:55:41.267 --> 01:55:42.602
EXPLORATION, YOUR SIX DAY

01:55:42.669 --> 01:55:44.437
EXPLORATION, WHERE YOU HAD TO

01:55:44.504 --> 01:55:46.673
EMPLOY AI YOURSELF TO DO SOME

01:55:46.739 --> 01:55:48.074
OF THAT UNDERSTANDING AND TO

01:55:48.141 --> 01:55:51.277
DIG INTO WHAT HAPPENED. COULD

01:55:51.344 --> 01:55:54.147
YOU DEFINE OR SHARE, IF THERE

01:55:54.213 --> 01:55:57.417
IS ONE, A SINGLE MOST IMPORTANT

01:55:57.483 --> 01:55:59.319
QUESTION OR A STANDOUT QUESTION

01:55:59.385 --> 01:56:01.387
THAT YOU WERE NOT ABLE TO

01:56:01.454 --> 01:56:03.823
INVESTIGATE, EITHER BECAUSE OF

01:56:03.890 --> 01:56:05.091
OR NOT ABLE TO REVEAL, BECAUSE

01:56:05.158 --> 01:56:07.126
OF REDACTION, BECAUSE OF SCOPE,

01:56:07.193 --> 01:56:08.928
BECAUSE OF TIME. BECAUSE OF

01:56:08.995 --> 01:56:10.730
CAPACITY. WHAT IS LINGERING

01:56:10.797 --> 01:56:12.231
STILL? SO I CAN GIVE AN EXAMPLE

01:56:12.298 --> 01:56:15.368
BASED ON SCOPE. SO IN THIS

01:56:15.435 --> 01:56:17.170
INCIDENT, WE HAD ACCESS TO THE

01:56:17.236 --> 01:56:19.339
MODEL TRANSCRIPTS. WE WERE NOT

01:56:19.405 --> 01:56:20.773
GIVEN ACCESS TO THE MODEL

01:56:20.840 --> 01:56:21.574
ITSELF. SO FOR INSTANCE WE

01:56:21.641 --> 01:56:23.676
COULD NOT GO BACK AND ASK THE

01:56:23.743 --> 01:56:25.445
MODEL QUESTIONS. WE WERE NOT

01:56:25.511 --> 01:56:27.647
ABLE TO GIVE THE MODEL MODIFIED

01:56:27.714 --> 01:56:28.748
VERSIONS OF THE TRANSCRIPTS AND

01:56:28.815 --> 01:56:30.149
SEE HOW IT REACTS. SO, FOR

01:56:30.216 --> 01:56:31.684
INSTANCE, WE WERE NOT ABLE TO

01:56:31.751 --> 01:56:32.752
GAUGE THE LIMITS OF THE

01:56:32.819 --> 01:56:34.120
BEHAVIOR OF THE MODEL. THAT

01:56:34.187 --> 01:56:35.621
WOULD BE AN EXAMPLE OF

01:56:35.688 --> 01:56:36.789
SOMETHING THAT WAS NOT IN SCOPE.

01:56:36.856 --> 01:56:39.525
SO IF I'M GOING TO DRAW A

01:56:39.592 --> 01:56:40.593
METAPHOR, THAT WOULD BE YOU

01:56:40.660 --> 01:56:42.362
COULD SEE PHOTOGRAPHS OF WHAT

01:56:42.428 --> 01:56:43.463
HAPPENED, BUT YOU COULDN'T GO

01:56:43.529 --> 01:56:45.998
BACK TO THE SCENE ITSELF TO

01:56:46.065 --> 01:56:47.800
LOOK AROUND AND GET A BETTER

01:56:47.867 --> 01:56:50.370
VIEW OF WHAT WAS HAPPENING.

01:56:50.436 --> 01:56:52.472
THAT SEEMS REASONABLE. IT WAS

01:56:52.538 --> 01:56:53.506
PRETTY GOOD PHOTOGRAPHS. WE DID

01:56:53.573 --> 01:56:54.640
HAVE VERY, VERY GOOD ACCESS

01:56:54.707 --> 01:56:55.708
FROM OPENAI, WHICH WE REALLY

01:56:55.775 --> 01:56:58.077
APPRECIATE. OKAY. I APPRECIATED

01:56:58.144 --> 01:56:59.379
YOU GIVING EXAMPLES OF SOME OF

01:56:59.445 --> 01:57:01.080
THE TRANSCRIPTS, THE LOG

01:57:01.147 --> 01:57:03.282
LANGUAGE. CERTAINLY, I SHARE

01:57:03.349 --> 01:57:05.084
CONCERNS THAT THE CHAIR HAS

01:57:05.151 --> 01:57:07.220
AROUND WHETHER IN THE FUTURE IN

01:57:07.286 --> 01:57:08.354
MODELS, THOSE LOGS ARE GOING TO

01:57:08.421 --> 01:57:10.056
BE UNDERSTANDABLE,

01:57:10.123 --> 01:57:11.958
COMPREHENSIBLE WITHIN HUMAN

01:57:12.024 --> 01:57:13.726
CAPACITY TO BE ABLE TO DECIPHER

01:57:13.793 --> 01:57:15.128
GOING FORWARD. SO APPRECIATE

01:57:15.194 --> 01:57:16.496
THAT AT LEAST IN THESE CASES,

01:57:16.562 --> 01:57:17.697
THERE WERE LOGS THAT WE COULD

01:57:17.764 --> 01:57:19.365
EXAMINE AND EXPLORE IN A

01:57:19.432 --> 01:57:20.533
LANGUAGE THAT WE COULD

01:57:20.600 --> 01:57:22.301
UNDERSTAND. YOU ALSO DISCLOSED

01:57:22.368 --> 01:57:24.303
WHILE DRAFTING THAT YOU WEIGHED

01:57:24.370 --> 01:57:26.639
THE POSSIBLE RISK OF YOUR VERY

01:57:26.706 --> 01:57:28.441
PRESENCE, YOUR VERY QUESTIONS,

01:57:28.508 --> 01:57:31.711
DISCOURAGING LABS FROM FUTURE

01:57:31.778 --> 01:57:33.946
PERMISSIVE INVESTIGATIONS. DID

01:57:34.013 --> 01:57:37.183
THAT LEAVE YOU? DID THAT CHANGE

01:57:37.250 --> 01:57:38.284
YOUR PERSPECTIVE IN YOUR

01:57:38.351 --> 01:57:40.787
APPROACH TO YOUR WORK AT ALL? I

01:57:40.853 --> 01:57:41.854
WAS NOT DIRECTLY INVOLVED WITH

01:57:41.921 --> 01:57:43.389
THE HUGGING FACE NEGOTIATIONS

01:57:43.456 --> 01:57:44.724
OR REPORT, SO I'M NOT ABLE TO

01:57:44.791 --> 01:57:47.860
ANSWER THAT. OKAY. THANK YOU.

01:57:47.927 --> 01:57:52.932
AND IS THERE? I GUESS YOU'RE

01:57:52.999 --> 01:57:54.400
NOT OFFERING ANY LEGISLATIVE

01:57:54.467 --> 01:57:55.568
ADVICE, SO I'LL HOLD THAT NEXT

01:57:55.635 --> 01:57:56.536
QUESTION FOR ANOTHER

01:57:56.602 --> 01:57:57.537
CONVERSATION. THANK YOU. HAVE A

01:57:57.603 --> 01:57:59.439
PANEL THAT WILL BE DELIGHTED TO

01:57:59.505 --> 01:58:01.974
DO THAT. SO YEAH, QUICK

01:58:02.041 --> 01:58:03.342
QUESTION THEN. SO I'M

01:58:03.409 --> 01:58:04.844
INTERESTED TO KNOW YOUR TAKE ON

01:58:04.911 --> 01:58:07.747
WHEN THE AI COMPANY IS SHOULD

01:58:07.814 --> 01:58:09.849
BE REQUIRED TO TELL A

01:58:09.916 --> 01:58:12.018
COMPROMISED COMPANY THAT THEIR

01:58:12.084 --> 01:58:15.321
AGENTS ATTACKED THEM. METER

01:58:15.388 --> 01:58:16.189
DOESN'T TAKE AN OPINION ON THAT.

01:58:16.255 --> 01:58:20.359
I'M SORRY. OKAY. THANK YOU. CAN

01:58:20.426 --> 01:58:22.595
YOU ANSWER WHEN THEY DO YOU

01:58:22.662 --> 01:58:24.497
THINK WHEN WOULD IT BE POSSIBLE

01:58:24.564 --> 01:58:28.968
FOR THEM TO TELL THEM? CAN YOU

01:58:29.035 --> 01:58:30.036
MAKE THE QUESTION MORE SPECIFIC?

01:58:30.102 --> 01:58:31.804
MAYBE. YEAH. USE POSSIBLE. OKAY.

01:58:31.871 --> 01:58:34.006
SO LIKE, OKAY, IF AN AI AGENT

01:58:34.073 --> 01:58:38.277
TRIES TO HACK A BOOKSTORE, HOW

01:58:38.344 --> 01:58:40.112
SOON WOULD IT BE POSSIBLE FOR

01:58:40.179 --> 01:58:43.316
THEM TO TELL THE BOOKSTORE?

01:58:43.382 --> 01:58:46.586
YEAH, IT'S QUITE HARD TO SAY.

01:58:46.652 --> 01:58:49.455
THE WAY THESE SYSTEMS WORK,

01:58:49.522 --> 01:58:51.257
ROUGHLY, IS THAT THE AGENTS ARE

01:58:51.324 --> 01:58:52.892
RUNNING AND THERE ARE MONITORS

01:58:52.959 --> 01:58:54.560
THAT ARE LOOKING AT THEM. THERE

01:58:54.627 --> 01:58:55.761
ARE QUESTIONS ABOUT WHETHER THE

01:58:55.828 --> 01:58:56.896
MONITOR IS GOING TO SEE WHAT

01:58:56.963 --> 01:58:58.731
IT'S LOOKING FOR OR NOT. THERE

01:58:58.798 --> 01:58:59.799
ARE ALSO QUESTIONS ABOUT THE

01:58:59.866 --> 01:59:01.367
LEVEL OF DETAIL. SO FOR EXAMPLE,

01:59:01.434 --> 01:59:03.503
IN THE OPENAI HUGGING FACE

01:59:03.569 --> 01:59:04.937
INCIDENT, OPENAI DID DISCLOSE

01:59:05.004 --> 01:59:07.406
TO THE PUBLIC A FEW DAYS AFTER

01:59:07.473 --> 01:59:10.376
HUGGING FACE ANNOUNCED I AM NOT

01:59:10.443 --> 01:59:11.410
INSIDE OPENAI OR ANY MAJOR

01:59:11.477 --> 01:59:13.012
FRONTIER COMPANY. I DON'T HAVE

01:59:13.079 --> 01:59:14.113
THE DETAILS OF HOW THEIR

01:59:14.180 --> 01:59:15.114
MONITORING SYSTEMS WORK, SO I

01:59:15.181 --> 01:59:17.016
CAN'T SAY IF THEY COULD HAVE

01:59:17.083 --> 01:59:18.251
DISCLOSED QUICKER OR NOT. I

01:59:18.317 --> 01:59:20.052
WILL POINT OUT THAT AT THE TIME

01:59:20.119 --> 01:59:21.020
OF THEIR INITIAL DISCLOSURE,

01:59:21.087 --> 01:59:22.188
THERE WAS STILL A LOT THAT

01:59:22.255 --> 01:59:24.023
WASN'T KNOWN. AND SO DOING A

01:59:24.090 --> 01:59:25.124
DEEP INVESTIGATION LIKE THE ONE

01:59:25.191 --> 01:59:26.526
WE DID TOOK SEVERAL WEEKS FOR

01:59:26.592 --> 01:59:31.597
US. THANK YOU. YES.

01:59:31.664 --> 01:59:32.198
ASSEMBLYMEMBER'S BIR AND THEN

01:59:32.265 --> 01:59:33.699
WE'LL COME TO YOU.

01:59:33.766 --> 01:59:36.969
ASSEMBLYMEMBERS. STEPHANIE.

01:59:37.036 --> 01:59:38.137
THANK YOU SO MUCH FOR BEING

01:59:38.204 --> 01:59:39.839
HERE TODAY. VERY INTERESTING

01:59:39.906 --> 01:59:42.008
PRESENTATION. AGAIN, I'VE BEEN

01:59:42.074 --> 01:59:43.276
SORT OF FOCUSED ON WHETHER

01:59:43.342 --> 01:59:47.413
THERE ARE THINGS THAT WE CAN DO.

01:59:47.480 --> 01:59:49.215
AT THE LEVEL OF THOSE WHO ARE

01:59:49.282 --> 01:59:51.751
DEPLOYING AI TO TRY TO PROTECT

01:59:51.817 --> 01:59:53.185
THE PUBLIC. SO, FOR EXAMPLE,

01:59:53.252 --> 01:59:54.387
ONE OF THE CALL, ONE OF THE

01:59:54.453 --> 01:59:56.289
SETS OF CALLS THAT I'VE BEEN

01:59:56.355 --> 01:59:58.257
GETTING FROM BOTH FINANCIAL

01:59:58.324 --> 01:59:59.292
INSTITUTIONS IN MY DISTRICT AS

01:59:59.358 --> 02:00:02.194
WELL AS PEOPLE, IS JUST A WORRY

02:00:02.261 --> 02:00:04.363
ABOUT THE SECURITY OF THEIR

02:00:04.430 --> 02:00:05.398
FINANCIAL RECORDS, THEIR

02:00:05.464 --> 02:00:06.465
ACCOUNTS IN BANKS, THEIR

02:00:06.532 --> 02:00:09.402
ACCOUNTS IN. IT SOUNDED TO ME

02:00:09.468 --> 02:00:11.537
LIKE ONE OF THE ONLY WAYS YOU

02:00:11.604 --> 02:00:13.039
COULD ACTUALLY PRETTY WELL TELL

02:00:13.105 --> 02:00:15.575
THEM IT SOUNDS LIKE ONE WAY YOU

02:00:15.641 --> 02:00:17.310
MIGHT POSSIBLY BE ABLE TO SORT

02:00:17.376 --> 02:00:21.280
OF PROTECT AGAINST THE HORRIFIC

02:00:21.347 --> 02:00:23.049
SCENARIO WHERE ALL OF A SUDDEN

02:00:23.115 --> 02:00:24.450
RECORDS ARE EITHER LOCKED OR

02:00:24.517 --> 02:00:27.320
BLOCKED OR VANISH, WOULD BE TO

02:00:27.386 --> 02:00:30.222
HAVE COPIES OF RECORDS

02:00:30.289 --> 02:00:31.791
PERIODICALLY DISCONNECTED FROM

02:00:31.857 --> 02:00:33.392
THE INTERNET. IS THAT SOMETHING

02:00:33.459 --> 02:00:36.762
THAT'S EVEN POSSIBLE? AND I

02:00:36.829 --> 02:00:39.599
JUST WANTED YEAH, I DO THINK

02:00:39.665 --> 02:00:40.800
THAT'S POSSIBLE. BUT I ALSO

02:00:40.866 --> 02:00:42.335
WANT TO BE CAREFUL THAT I

02:00:42.401 --> 02:00:44.236
DIDN'T CAUSE A STAMPEDE BY

02:00:44.303 --> 02:00:46.005
MAYBE MISSPEAKING. I GAVE THAT

02:00:46.072 --> 02:00:48.641
AS AN EXTREME EXAMPLE. IT

02:00:48.708 --> 02:00:49.842
WASN'T MEANT TO BE A SUGGESTION.

02:00:49.909 --> 02:00:51.711
IN THE SAME WAY THAT WE CAN

02:00:51.777 --> 02:00:52.745
AVOID SOME OF LIFE'S

02:00:52.812 --> 02:00:55.548
DIFFICULTIES BY NEVER GOING

02:00:55.615 --> 02:00:59.752
OUTSIDE. SECOND, SECOND

02:00:59.819 --> 02:01:03.389
QUESTION YOU'RE TALKING ABOUT

02:01:03.456 --> 02:01:06.993
YOU PROVIDE A SORT OF A, AN

02:01:07.059 --> 02:01:08.394
ACTIVITY THAT IS ESSENTIALLY

02:01:08.461 --> 02:01:10.196
SORT OF AN INDEPENDENT

02:01:10.262 --> 02:01:12.732
ASSESSMENT, AND THAT'S DONE AT

02:01:12.798 --> 02:01:16.268
THE, AT THE, AT THE LEVEL OF

02:01:16.335 --> 02:01:19.939
THE, THE COMPANIES THAT ARE

02:01:20.006 --> 02:01:22.074
PRODUCING THE AI MODELS. YES.

02:01:22.141 --> 02:01:25.344
IS THERE IS THERE WOULD IT BE.

02:01:25.411 --> 02:01:26.579
SO FIRST OF ALL, WHAT ARE THE

02:01:26.646 --> 02:01:28.347
SORT OF THE STANDARDS YOU'RE

02:01:28.414 --> 02:01:29.515
USING? I THINK YOU SAID THAT

02:01:29.582 --> 02:01:31.384
THOSE WERE SORT OF EMERGING NOW

02:01:31.450 --> 02:01:32.485
IN TERMS OF DOING THOSE

02:01:32.551 --> 02:01:34.286
ASSESSMENTS. BUT WOULD IT BE

02:01:34.353 --> 02:01:35.721
PRUDENT TO EXPLORE WHETHER OR

02:01:35.788 --> 02:01:37.890
NOT ASSESSMENTS SHOULD BE DONE

02:01:37.957 --> 02:01:39.458
AT THE LEVEL OF COMPANIES THAT

02:01:39.525 --> 02:01:42.428
ARE DEPLOYING THE DEPLOYING

02:01:42.495 --> 02:01:49.502
THESE MODELS? I'M NOT SURE.

02:01:49.568 --> 02:01:50.936
THANK YOU. THAT'S WHY I DON'T

02:01:51.003 --> 02:01:51.904
SLEEP AT NIGHT, BECAUSE THERE'S

02:01:51.971 --> 02:01:53.639
A LOT THAT WE DON'T KNOW.

02:01:53.706 --> 02:01:54.507
ASSEMBLYMEMBER'S STEPHANIE.

02:01:54.573 --> 02:01:55.941
THANK YOU. I'M NEVER GOING TO

02:01:56.008 --> 02:01:59.612
SLEEP AGAIN ACTUALLY. SO THANK

02:01:59.679 --> 02:02:00.980
YOU FOR BEING HERE. AND THANK

02:02:01.047 --> 02:02:02.548
YOU, CHAIR, FOR THIS HEARING.

02:02:02.615 --> 02:02:03.883
AND THANK YOU FOR INVITING ME

02:02:03.949 --> 02:02:05.618
TO IT, SINCE I'M NOT ON THE

02:02:05.685 --> 02:02:06.852
MEMBER OF THE MEMBER OF THIS

02:02:06.919 --> 02:02:10.356
COMMITTEE, I JUST I WANT TO GET

02:02:10.423 --> 02:02:14.593
CLEAR IN MY MIND. SO DO AI

02:02:14.660 --> 02:02:16.762
AGENTS ALWAYS OPERATE AT THE

02:02:16.829 --> 02:02:23.302
PROMPT OF A HUMAN? NO. OKAY. SO

02:02:23.369 --> 02:02:25.805
WHAT PUTS THEM INTO FORCE IN

02:02:25.871 --> 02:02:27.606
TERMS OF A TASK, LIKE YOU

02:02:27.673 --> 02:02:29.341
EXPLAINED WITH HUGGING FACE?

02:02:29.408 --> 02:02:33.379
WELL, THE AGENT HAS TO HAVE

02:02:33.446 --> 02:02:35.147
SOME PROMPT OR SOME WAY OF

02:02:35.214 --> 02:02:37.717
ACTING, BUT THEY ARE OFTEN ABLE

02:02:37.783 --> 02:02:39.485
TO. SO ONE SIMPLE EXAMPLE WOULD

02:02:39.552 --> 02:02:41.287
BE THAT IF YOU ASK AN AGENT TO

02:02:41.353 --> 02:02:42.822
DO SOMETHING, IT CAN OFTEN

02:02:42.888 --> 02:02:44.190
INVOKE A SUB AGENT AND IT CAN

02:02:44.256 --> 02:02:46.392
PROMPT THAT SUB AGENT. AND SO

02:02:46.459 --> 02:02:47.493
THE PROMPT FOR THE SUB AGENT

02:02:47.560 --> 02:02:48.961
CAME DIRECTLY FROM THE FIRST

02:02:49.028 --> 02:02:50.162
AGENT, NOT FROM THE HUMAN, BUT

02:02:50.229 --> 02:02:55.301
WHO DOES THE INITIAL ASKING.

02:02:55.367 --> 02:02:57.837
WHO IN OPENAI, WHO IS DOING THE

02:02:57.903 --> 02:03:00.673
INITIAL ASKING? YEAH, MY

02:03:00.740 --> 02:03:01.841
UNDERSTANDING, AND AGAIN, I'M

02:03:01.907 --> 02:03:03.275
NOT A FULL EXPERT ON THIS, IS

02:03:03.342 --> 02:03:06.479
THAT THESE THINGS ARE DEPLOYED

02:03:06.545 --> 02:03:13.686
AUTOMATICALLY. SO THERE'S A.

02:03:13.753 --> 02:03:14.920
I'M GOING TO HAVE TO GET BACK

02:03:14.987 --> 02:03:16.956
TO YOU ON THIS QUESTION. IT'S I

02:03:17.022 --> 02:03:21.227
WHO INVENTED AI. IT DIDN'T

02:03:21.293 --> 02:03:23.729
INVENT ITSELF. AI WAS INVENTED

02:03:23.796 --> 02:03:25.531
BY A LARGE NUMBER OF PEOPLE

02:03:25.598 --> 02:03:26.632
WORKING COLLECTIVELY OVER

02:03:26.699 --> 02:03:29.535
DECADES. HUMANS. YES. OKAY. AND

02:03:29.602 --> 02:03:32.371
SO WHEN AN AI AGENT ALTERS ITS

02:03:32.438 --> 02:03:34.206
BEHAVIOR TO AVOID DETECTION,

02:03:34.273 --> 02:03:36.742
IT'S RESPONDING TO A SERIES OF

02:03:36.809 --> 02:03:38.911
STRICTLY LOGICAL AND STRUCTURAL

02:03:38.978 --> 02:03:40.346
DRIVERS CREATED BY HUMANS.

02:03:40.412 --> 02:03:42.915
CORRECT. I'M NOT SURE I WOULD

02:03:42.982 --> 02:03:44.283
CHOOSE TO SAY THEY'RE STRICTLY

02:03:44.350 --> 02:03:46.452
LOGICAL. WELL, OKAY, TAKE OUT

02:03:46.519 --> 02:03:50.022
THE WORD LOGICAL JUST BY

02:03:50.089 --> 02:03:52.958
DRIVERS. WHAT I'M TRYING TO GET

02:03:53.025 --> 02:03:55.427
AT IS I DON'T THINK LIKE AN AI

02:03:55.494 --> 02:03:56.529
DEVELOPER. I DON'T THINK LIKE A

02:03:56.595 --> 02:03:58.230
SOFTWARE. I THINK LIKE A LAWYER.

02:03:58.297 --> 02:03:59.398
AND I'M THINKING ABOUT

02:03:59.465 --> 02:04:01.200
CAUSATION AND I'M THINKING

02:04:01.267 --> 02:04:02.635
ABOUT FORESEEABILITY AND

02:04:02.701 --> 02:04:04.436
REASONABLE FORESEEABILITY. AND

02:04:04.503 --> 02:04:06.238
WHEN I HAVE ASSEMBLY MEMBER MIA

02:04:06.305 --> 02:04:08.440
BONTA ASKING YOU WHETHER OR NOT

02:04:08.507 --> 02:04:13.078
ANYONE COULD PUT INTO A DRIVER

02:04:13.145 --> 02:04:15.247
ANY TYPE OF HUMANE BEHAVIOR,

02:04:15.314 --> 02:04:17.049
AND YOUR ANSWER BACK IS, I

02:04:17.116 --> 02:04:20.653
DON'T KNOW OR NO, THAT'S

02:04:20.719 --> 02:04:23.155
TERRIFYING. SO I'M TRYING TO

02:04:23.222 --> 02:04:24.990
UNDERSTAND WHO'S PUTTING IT IN

02:04:25.057 --> 02:04:27.493
MOTION SO I CAN THEN HOLD THAT

02:04:27.560 --> 02:04:30.763
PERSON ACCOUNTABLE WHEN HARM IS

02:04:30.830 --> 02:04:32.264
CAUSED. THAT'S WHERE I'M GOING

02:04:32.331 --> 02:04:36.535
WITH THIS. THANK YOU. I THINK I

02:04:36.602 --> 02:04:39.004
MEAN LAWYER TO LAWYER, ASSEMBLY

02:04:39.071 --> 02:04:42.341
MEMBER. I THINK THERE IS. AND

02:04:42.408 --> 02:04:43.809
CORRECT ME IF I GUESS THIS

02:04:43.876 --> 02:04:45.044
WOULD BE THE FOLLOW UP QUESTION,

02:04:45.110 --> 02:04:47.246
WHICH IS IT IS FORESEEABLE THAT

02:04:47.313 --> 02:04:49.114
IN THE FUTURE THE AI COULD SPIN

02:04:49.181 --> 02:04:51.217
OUT AGENTS AND THE AI, THE

02:04:51.283 --> 02:04:52.418
LARGE LANGUAGE MODEL ITSELF

02:04:52.484 --> 02:04:54.086
COULD PROMPT AN AGENT TO ACT

02:04:54.153 --> 02:04:55.721
WITHOUT HUMAN PROMPT. I DON'T

02:04:55.788 --> 02:04:57.690
SEE WHY THAT'S NOT POSSIBLE.

02:04:57.756 --> 02:04:58.924
LARGE LANGUAGE MODELS PROMPT

02:04:58.991 --> 02:05:00.826
EACH OTHER TO ACT ALL THE TIME.

02:05:00.893 --> 02:05:01.961
ALL THE TIME ALREADY? YES, BUT

02:05:02.027 --> 02:05:03.128
THE BUT THE LARGE LANGUAGE

02:05:03.195 --> 02:05:04.897
MODEL ITSELF WAS CREATED BY THE

02:05:04.964 --> 02:05:06.799
LABS AND BY HUMANS AT THE LAB,

02:05:06.866 --> 02:05:07.833
FUNDAMENTALLY. AND SO DESPITE

02:05:07.900 --> 02:05:09.235
THE FACT THAT THE LLMS AND

02:05:09.301 --> 02:05:10.970
AGENTS AT THIS POINT MIGHT BE

02:05:11.036 --> 02:05:12.238
ACTING WITHOUT HUMAN PROMPT, AT

02:05:12.304 --> 02:05:14.006
THE BOTTOM OF THAT IS THE

02:05:14.073 --> 02:05:15.808
HUMANS AT THE LAB, CORRECT?

02:05:15.875 --> 02:05:18.911
YEAH. OKAY. BUT HE'S A LAWYER,

02:05:18.978 --> 02:05:21.881
SO HE'S LIKE, OKAY, ANYBODY

02:05:21.947 --> 02:05:24.383
ELSE? YEAH. OKAY. SO MEMBER

02:05:24.450 --> 02:05:25.985
SCHULZ AND THEN WE'LL COME TO

02:05:26.051 --> 02:05:28.687
ERWIN. OKAY. SO REMEMBER ERWIN

02:05:28.754 --> 02:05:30.155
THÀNH ASSEMBLYMEMBERS SCHULZ

02:05:30.222 --> 02:05:31.223
HOPEFULLY QUICK, YOU SAID YOU

02:05:31.290 --> 02:05:34.159
HAD THREE RESEARCHERS FOR SIX

02:05:34.226 --> 02:05:36.595
DAYS, WHICH SEEMS LIKE A

02:05:36.662 --> 02:05:38.030
RELATIVELY QUICK TIME TO

02:05:38.097 --> 02:05:39.465
RECONSTRUCT WHAT HAPPENED. THIS

02:05:39.531 --> 02:05:43.102
LOOKS VERY COMPLICATED. WOULD

02:05:43.168 --> 02:05:44.904
THE COMPANY HAVE HAD THE SAME

02:05:44.970 --> 02:05:47.907
CAPACITY AS METER? I'M NOT ABLE

02:05:47.973 --> 02:05:50.342
TO ANSWER THAT. OKAY. SO DO

02:05:50.409 --> 02:05:51.877
THEY AGREE WITH YOUR ASSESSMENT

02:05:51.944 --> 02:05:55.381
OF WHAT HAPPENED? OPENAI HAD

02:05:55.447 --> 02:05:57.516
THE RIGHT TO REDACT THINGS FROM

02:05:57.583 --> 02:05:58.884
OUR REPORT, AND THEY APPROVED

02:05:58.951 --> 02:06:02.154
OF ITS PUBLICATION. OKAY. WHEN

02:06:02.221 --> 02:06:03.956
WHEN WE'RE TALKING ABOUT

02:06:04.023 --> 02:06:06.091
INDEPENDENT ASSESSMENTS OF

02:06:06.158 --> 02:06:08.994
THESE THESE COMPANIES, HOW MANY

02:06:09.061 --> 02:06:10.763
COMPANIES ARE OF THESE LLMS?

02:06:10.829 --> 02:06:12.598
HOW MANY ARE THERE THAT ARE

02:06:12.665 --> 02:06:15.134
ABLE TO PERFORM THE TASK THAT

02:06:15.200 --> 02:06:19.104
METER DOES? I'M NOT SURE. AND

02:06:19.171 --> 02:06:21.407
SO THE I THINK THE QUESTION

02:06:21.473 --> 02:06:23.409
THAT I'VE SEEN A LOT, YOU KNOW,

02:06:23.475 --> 02:06:25.878
IN THESE TECHNICAL DISCUSSIONS

02:06:25.945 --> 02:06:30.215
IS WHETHER METER IS ABLE TO ACT

02:06:30.282 --> 02:06:31.183
INDEPENDENTLY WHEN THEY DO

02:06:31.250 --> 02:06:32.618
THESE INDEPENDENT ASSESSMENTS,

02:06:32.685 --> 02:06:34.586
IF THEY'RE THE ONLY COMPANY

02:06:34.653 --> 02:06:36.689
CAPABLE OF DOING THEM, OR IS

02:06:36.755 --> 02:06:38.190
THERE A BIG ENOUGH WORKFORCE TO

02:06:38.257 --> 02:06:40.292
ACTUALLY ASSESS THESE? YEAH,

02:06:40.359 --> 02:06:41.393
THERE ARE A FEW GOOD QUESTIONS

02:06:41.460 --> 02:06:42.962
THERE, AND I'LL ADDRESS THEM

02:06:43.028 --> 02:06:45.397
ONE BY ONE. IN TERMS OF

02:06:45.464 --> 02:06:46.365
INDEPENDENCE, THERE ARE SOME

02:06:46.432 --> 02:06:47.566
REAL CHALLENGES HERE. WE NEED

02:06:47.633 --> 02:06:48.968
TO MAINTAIN OUR RELATIONSHIPS

02:06:49.034 --> 02:06:50.903
WITH THE LABS, AND THAT MEANS

02:06:50.970 --> 02:06:52.905
THAT WE DEFINITELY FEEL SOFT

02:06:52.972 --> 02:06:55.107
PRESSURE TO ACT IN A WAY THAT

02:06:55.174 --> 02:06:56.508
THE LABS WILL WANT TO CONTINUE

02:06:56.575 --> 02:06:59.812
WORKING WITH US AND CONTINUE TO

02:06:59.878 --> 02:07:01.613
GIVE US ACCESS, RIGHT? WE ARE

02:07:01.680 --> 02:07:02.915
DEPENDENT ON ACCESS TO THE LABS

02:07:02.982 --> 02:07:04.883
IN ORDER TO DO OUR WORK. AND SO

02:07:04.950 --> 02:07:06.051
THAT DOES CREATE PRESSURES. WE

02:07:06.118 --> 02:07:07.820
TRY TO BE AWARE OF THOSE

02:07:07.886 --> 02:07:08.921
PRESSURES AND TO NAVIGATE THEM

02:07:08.988 --> 02:07:10.923
AS BEST WE CAN. WE TRY TO HAVE

02:07:10.990 --> 02:07:12.024
PRINCIPLES ABOUT WHAT WE'LL SAY

02:07:12.091 --> 02:07:13.225
AND TO BASICALLY ALWAYS SPEAK

02:07:13.292 --> 02:07:14.927
THE TRUTH AND TO ALWAYS SAY

02:07:14.994 --> 02:07:16.028
WHAT'S IMPORTANT. BUT THERE'S A

02:07:16.095 --> 02:07:17.129
REAL CHALLENGE THERE, FOR SURE.

02:07:17.196 --> 02:07:18.931
THE SECOND POINT YOU MADE WAS

02:07:18.998 --> 02:07:20.032
ABOUT THE WORKFORCE. I DO THINK

02:07:20.099 --> 02:07:21.867
THE TOTAL WORKFORCE HERE IS

02:07:21.934 --> 02:07:23.135
CURRENTLY FAIRLY LIMITED. THIS

02:07:23.202 --> 02:07:25.371
WORK IS EXTREMELY DIFFICULT AND

02:07:25.437 --> 02:07:27.139
REQUIRES HIGH LEVEL SKILLS.

02:07:27.206 --> 02:07:28.707
THERE ARE NOT A TON OF PEOPLE

02:07:28.774 --> 02:07:30.676
WHO CAN DO IT. WE ARE WORKING

02:07:30.743 --> 02:07:33.278
TO MAKE IT EASIER TO TRAIN MORE

02:07:33.345 --> 02:07:34.113
PEOPLE. WE'RE TRYING TO TRAIN

02:07:34.179 --> 02:07:35.814
PEOPLE UP INTERNALLY. WE WOULD

02:07:35.881 --> 02:07:37.082
LOVE TO BE ABLE TO TRAIN PEOPLE

02:07:37.149 --> 02:07:38.917
OUTSIDE OUR ORG TO DO IT. WE

02:07:38.984 --> 02:07:40.786
WOULD LOVE TO SEE A THRIVING

02:07:40.853 --> 02:07:41.920
GROUP OF THIRD PARTIES WHO CAN

02:07:41.987 --> 02:07:44.490
ALL DO THIS. I THINK THAT SO

02:07:44.556 --> 02:07:46.225
THE WORKFORCE IS A BIG ISSUE,

02:07:46.291 --> 02:07:48.027
AND WE SHOULD BE LOOKING AT

02:07:48.093 --> 02:07:50.195
UPSKILLING SOME OF THE SOFTWARE

02:07:50.262 --> 02:07:51.964
ENGINEERS MAYBE THAT CAN'T FIND

02:07:52.031 --> 02:07:53.999
FIND JOBS RIGHT NOW. OBVIOUSLY,

02:07:54.066 --> 02:07:55.234
THERE IS A BIG INTEREST FOR THE

02:07:55.300 --> 02:07:57.236
THE BIG COMPANIES TO BE DOING

02:07:57.302 --> 02:07:58.203
THESE INTERNAL ASSESSMENTS AND

02:07:58.270 --> 02:08:03.342
TO BE ABLE TO, TO. LET THE

02:08:03.409 --> 02:08:05.177
PUBLIC KNOW THAT THEY ARE

02:08:05.244 --> 02:08:06.945
CONCERNED ABOUT SAFETY. WHAT

02:08:07.012 --> 02:08:08.747
ABOUT ALL THESE OPEN SOURCE,

02:08:08.814 --> 02:08:10.916
OPEN WEIGHT MODELS? ARE SOME OF

02:08:10.983 --> 02:08:13.852
THOSE COMPANIES ALSO USING YOUR

02:08:13.919 --> 02:08:15.988
SERVICES? BECAUSE CERTAINLY WE

02:08:16.055 --> 02:08:18.123
WOULD LIKE TO MAKE SURE THAT

02:08:18.190 --> 02:08:19.825
THEY ARE FUNCTIONING IN A SAFE

02:08:19.892 --> 02:08:22.027
WAY ALSO. YEAH, IT'S A GREAT

02:08:22.094 --> 02:08:23.529
QUESTION. WE'RE IN PRINCIPLE,

02:08:23.595 --> 02:08:25.364
HIGHLY OPEN TO WORKING WITH

02:08:25.431 --> 02:08:27.866
OPEN SOURCE COMPANIES OR CLOSED

02:08:27.933 --> 02:08:29.234
LABS, THE FRONTIER LABS. IT

02:08:29.301 --> 02:08:30.636
DOESN'T MAKE A DIFFERENCE TO US

02:08:30.702 --> 02:08:32.337
IN PRINCIPLE DUE TO CAPACITY.

02:08:32.404 --> 02:08:33.639
WE DEFINITELY PRIORITIZE

02:08:33.705 --> 02:08:34.940
WORKING WITH THE COMPANIES THAT

02:08:35.007 --> 02:08:36.475
HAVE THE MOST ADVANCED MODELS,

02:08:36.542 --> 02:08:37.943
BECAUSE THAT'S WHERE WE THINK

02:08:38.010 --> 02:08:39.745
THE Kò PHENOMENA WILL SHOW UP

02:08:39.812 --> 02:08:41.413
FIRST. AND SO THAT'S WHERE WE

02:08:41.480 --> 02:08:43.348
GO TO STUDY THEM. ALL RIGHT.

02:08:43.415 --> 02:08:44.683
OKAY. THANK YOU. ALTHOUGH I'LL

02:08:44.750 --> 02:08:46.552
NOTE THAT IF IT WAS TRULY OPEN

02:08:46.618 --> 02:08:48.454
WEIGHT, AND THIS IS A LITTLE

02:08:48.520 --> 02:08:49.588
BIT OF CONFUSION IN THIS SPACE,

02:08:49.655 --> 02:08:51.323
OPEN SOURCE VERSUS OPEN WEIGHT,

02:08:51.390 --> 02:08:53.058
YOU COULD DO THIS WITHOUT THE

02:08:53.125 --> 02:08:54.159
COMPANY PROVIDING ACCESS

02:08:54.226 --> 02:08:54.960
BECAUSE YOU WOULD HAVE OPEN

02:08:55.027 --> 02:08:56.462
ACCESS TO THE INFORMATION TO DO

02:08:56.528 --> 02:08:58.497
THIS RESEARCH. ISN'T THAT

02:08:58.564 --> 02:09:02.167
CORRECT? I THINK SO, POSSIBLY,

02:09:02.234 --> 02:09:03.502
ALTHOUGH IT'S QUITE COMPLEX.

02:09:03.569 --> 02:09:04.470
RIGHT. I KNOW, BECAUSE THEY'RE

02:09:04.536 --> 02:09:06.038
NOT ALL ACTUALLY PROVIDING FULL

02:09:06.105 --> 02:09:08.540
OPENNESS AS PART OF THE COMPLEX

02:09:08.607 --> 02:09:09.408
NATURE OF IT. BUT THAT'S THE

02:09:09.475 --> 02:09:11.143
IDEA BEHIND OPEN WEIGHTS. IS

02:09:11.210 --> 02:09:12.144
THAT BECAUSE IT'S PUBLICLY

02:09:12.211 --> 02:09:13.312
AVAILABLE, WE HAVE MORE INSIGHT.

02:09:13.378 --> 02:09:15.380
AND YEAH, ALTHOUGH WITHOUT FULL

02:09:15.447 --> 02:09:17.516
OPEN SOURCE, JUST OPEN WEIGHTS

02:09:17.583 --> 02:09:19.785
DOESN'T MAKE IT VERY EASY TO DO

02:09:19.852 --> 02:09:21.019
THE DEEPEST EXPERIMENTS YOU

02:09:21.086 --> 02:09:22.788
MIGHT WANT TO DO. GOT IT. SO

02:09:22.855 --> 02:09:23.989
YOU WOULD NEED BOTH. YOU WOULD

02:09:24.056 --> 02:09:25.958
NEED IT TO BE FULLY OPEN. YEAH.

02:09:26.024 --> 02:09:29.461
OPEN SOURCE. YES. SORRY. YES.

02:09:29.528 --> 02:09:33.732
THANK YOU. YOU SAID OPEN AI HAD

02:09:33.799 --> 02:09:36.235
THE RIGHT TO REDACT AND

02:09:36.301 --> 02:09:37.369
APPROVED THE PUBLICATION OF

02:09:37.436 --> 02:09:40.005
THIS REPORT. DID THEY EXERCISE

02:09:40.072 --> 02:09:41.673
THEIR RIGHT TO REDACT

02:09:41.740 --> 02:09:43.909
INFORMATION? I BELIEVE THERE

02:09:43.976 --> 02:09:45.277
ARE THE DETAILS OF THAT ARE IN

02:09:45.344 --> 02:09:47.346
THE REPORT. I'D HAVE TO LOOK AT

02:09:47.412 --> 02:09:53.886
IT AND GET BACK TO YOU. OKAY.

02:09:53.952 --> 02:09:55.053
YEAH. I BELIEVE PETER SAID IN

02:09:55.120 --> 02:09:57.122
THE REPORT THAT THERE WERE

02:09:57.189 --> 02:09:58.190
REDACTIONS, BUT NOTHING THAT

02:09:58.257 --> 02:09:59.391
ALTERED THE CONCLUSIONS IN THE

02:09:59.458 --> 02:10:00.526
REPORT WAS WHAT WAS IN THE

02:10:00.592 --> 02:10:02.895
REPORT. FOR WHAT IT'S WORTH.

02:10:02.961 --> 02:10:04.429
ASSEMBLYMEMBER'S SCHULTZ, THANK

02:10:04.496 --> 02:10:07.266
YOU VERY MUCH, MADAM CHAIR. SO,

02:10:07.332 --> 02:10:08.700
SIR, THANK YOU FOR YOUR

02:10:08.767 --> 02:10:10.102
PRESENTATION. I JUST WANTED TO

02:10:10.169 --> 02:10:11.503
CLARIFY A COUPLE QUICK THINGS.

02:10:11.570 --> 02:10:13.338
IN YOUR PRESENTATION, YOU

02:10:13.405 --> 02:10:15.841
TALKED ABOUT THE INVESTIGATION

02:10:15.908 --> 02:10:16.909
LIMITATIONS. I BELIEVE THE

02:10:16.975 --> 02:10:18.043
FIRST THING YOU LISTED ON THE

02:10:18.110 --> 02:10:20.179
SLIDE WAS THE NARROW SCOPE. I

02:10:20.245 --> 02:10:23.382
JUST WANTED TO CONFIRM. SO THE

02:10:23.448 --> 02:10:24.850
INVESTIGATION DID NOT EVALUATE

02:10:24.917 --> 02:10:26.018
WHETHER THE INCIDENT WAS PART

02:10:26.084 --> 02:10:27.786
OF A WIDER PATTERN OF DANGEROUS

02:10:27.853 --> 02:10:28.921
MODEL BEHAVIOR. CORRECT? THAT

02:10:28.987 --> 02:10:30.923
IS CORRECT. AND IT DID NOT

02:10:30.989 --> 02:10:32.391
ASSESS HOW THE BEHAVIOR HAD

02:10:32.457 --> 02:10:34.226
ARISEN DURING TRAINING. THAT IS

02:10:34.293 --> 02:10:36.662
CORRECT. OR WHETHER OPENAI

02:10:36.728 --> 02:10:38.897
SAFEGUARDS WERE ADEQUATE. THAT

02:10:38.964 --> 02:10:40.666
IS CORRECT. AND I JUST WANT TO

02:10:40.732 --> 02:10:41.700
BE CLEAR, THAT'S NOT

02:10:41.767 --> 02:10:42.834
NECESSARILY AT THE DIRECTION OF

02:10:42.901 --> 02:10:44.303
THE INVESTIGATORS. THAT WAS A

02:10:44.369 --> 02:10:45.704
PARAMETER OF THE VOLUNTARY

02:10:45.771 --> 02:10:47.172
AGREEMENT ENTERED INTO WITH

02:10:47.239 --> 02:10:51.109
OPENAI. YES. OKAY. THE SECOND

02:10:51.176 --> 02:10:52.945
THING I WANTED TO ASK WAS I

02:10:53.011 --> 02:10:54.780
ALSO NOTED IN READING THE

02:10:54.846 --> 02:10:55.814
MATERIALS THAT INVESTIGATORS

02:10:55.881 --> 02:10:56.882
WERE PREVENTED FROM RUNNING

02:10:56.949 --> 02:10:57.983
EXPERIMENTS ON THE INTERNAL

02:10:58.050 --> 02:11:00.485
MODEL. YOU TESTIFIED TO THAT

02:11:00.552 --> 02:11:01.954
THAT WASN'T BECAUSE THE

02:11:02.020 --> 02:11:02.988
INVESTIGATORS DIDN'T HAVE AN

02:11:03.055 --> 02:11:04.489
INTEREST IN TAKING THAT

02:11:04.556 --> 02:11:05.524
INVESTIGATIVE APPROACH. IT'S

02:11:05.591 --> 02:11:06.792
BECAUSE THEY WERE PRECLUDED

02:11:06.858 --> 02:11:08.660
FROM DOING SO. UNDER THE TERMS

02:11:08.727 --> 02:11:11.029
OF THE VOLUNTARY AGREEMENT, AS

02:11:11.096 --> 02:11:12.164
I UNDERSTAND IT, THAT WOULD

02:11:12.231 --> 02:11:13.999
HAVE BEEN OUT OF SCOPE FOR THE

02:11:14.066 --> 02:11:15.634
VOLUNTARY AGREEMENT. OKAY. AND

02:11:15.701 --> 02:11:16.902
I GUESS THE OTHER QUESTION I

02:11:16.969 --> 02:11:18.503
HAD TO CLOSE ON WAS WHAT MISS

02:11:18.570 --> 02:11:20.305
BONTE HAD ASKED. BUT I SUPPOSE

02:11:20.372 --> 02:11:21.540
I WOULD JUST SAY, FOR THE SAKE

02:11:21.607 --> 02:11:23.408
OF YOUR TESTIMONY TODAY, AS YOU

02:11:23.475 --> 02:11:24.977
SIT HERE RIGHT NOW, ARE YOU

02:11:25.043 --> 02:11:26.044
AWARE OF WHETHER ANY

02:11:26.111 --> 02:11:26.845
INFORMATION WAS REDACTED FROM

02:11:26.912 --> 02:11:31.116
THE REPORT? I BELIEVE THAT SOME

02:11:31.183 --> 02:11:33.986
INFORMATION WAS REDACTED AROUND

02:11:34.052 --> 02:11:35.387
QUOTES FROM THE TRANSCRIPTS OF

02:11:35.454 --> 02:11:41.493
AGENTS. OKAY. THANK YOU. THANK

02:11:41.560 --> 02:11:49.468
YOU. YES. THANK YOU. ONE OF THE

02:11:49.534 --> 02:11:53.038
MAMA BEARS WHO JUMPED IN ON THE

02:11:53.105 --> 02:11:54.206
JUMPED IN ON THE COMMITTEE.

02:11:54.273 --> 02:11:56.608
THANK YOU SO MUCH TO THE CHAIR

02:11:56.675 --> 02:11:58.477
FOR MAKING SPACE FOR ME AND ALL

02:11:58.543 --> 02:12:01.313
OF US. THANK YOU SO MUCH FOR

02:12:01.380 --> 02:12:02.781
THE WORK THAT YOU'RE DOING AND,

02:12:02.848 --> 02:12:04.283
AND FOR THE TRANSPARENCY. I

02:12:04.349 --> 02:12:06.351
THINK IT'S SO CRITICAL FOR THE

02:12:06.418 --> 02:12:07.753
WORK THAT WE ARE ATTEMPTING TO

02:12:07.819 --> 02:12:12.457
DO HERE. AND JUST SO I

02:12:12.524 --> 02:12:16.061
UNDERSTAND, IF IN AN EXTREME

02:12:16.128 --> 02:12:18.530
SCENARIO, YOU KNOW, WE WERE

02:12:18.597 --> 02:12:20.132
LIKE, SHUT EVERYTHING DOWN, YOU

02:12:20.198 --> 02:12:21.867
GOT TO STOP THIS UNTIL WE

02:12:21.933 --> 02:12:24.736
FIGURE IT OUT. IT'S IT'S NOT A

02:12:24.803 --> 02:12:25.837
SITUATION WHERE YOU CAN EVEN

02:12:25.904 --> 02:12:27.572
PUT EVERYTHING BACK INTO THE

02:12:27.639 --> 02:12:29.374
BOX AT THIS POINT, RIGHT? I

02:12:29.441 --> 02:12:30.876
MEAN, YOU'RE TALKING ABOUT AI

02:12:30.942 --> 02:12:32.244
BEING ABLE TO REPLICATE ITSELF

02:12:32.311 --> 02:12:35.180
ON IN DATA CENTERS. AND IS

02:12:35.247 --> 02:12:38.450
THERE EVEN A WAY, I THINK

02:12:38.517 --> 02:12:40.619
THERE'S A QUESTION ABOUT WHERE

02:12:40.686 --> 02:12:42.754
IN TIME YOU'RE TALKING ABOUT.

02:12:42.821 --> 02:12:44.656
AS OF TODAY, WE DO NOT BELIEVE

02:12:44.723 --> 02:12:46.325
THAT AI CAN REPLICATE

02:12:46.391 --> 02:12:48.126
THEMSELVES IN DATA CENTERS.

02:12:48.193 --> 02:12:51.396
OKAY. SO THEORETICALLY IT COULD

02:12:51.463 --> 02:12:54.199
BE SHUT DOWN. AS OF TODAY, IT

02:12:54.266 --> 02:12:58.537
SEEMS LIKE IT IS POSSIBLE. OKAY.

02:12:58.603 --> 02:13:00.105
AND IN TERMS OF THE WORK THAT

02:13:00.172 --> 02:13:03.575
YOU ALL DO, CAN YOU CAN YOU

02:13:03.642 --> 02:13:04.743
EXPLAIN A LITTLE BIT MORE ABOUT

02:13:04.810 --> 02:13:06.611
HOW YOU DO YOUR WORK? SO YOU

02:13:06.678 --> 02:13:10.782
HAVE TO BE INVITED IN BY A LAB

02:13:10.849 --> 02:13:14.019
OR COMPANY, OR CAN YOU REQUEST

02:13:14.086 --> 02:13:16.254
TO INVESTIGATE SOMETHING? I

02:13:16.321 --> 02:13:17.522
KNOW I DON'T THINK IT'S BEEN

02:13:17.589 --> 02:13:18.724
RAISED YET, BUT THERE'S BEEN A

02:13:18.790 --> 02:13:20.625
LOT OF TALK THIS WEEK ABOUT HOW

02:13:20.692 --> 02:13:24.830
AI BROKE THE INTERNET AND IS,

02:13:24.896 --> 02:13:26.264
YOU KNOW, AND CONCERNS ABOUT

02:13:26.331 --> 02:13:28.066
GETTING INTO BANK ACCOUNTS. AND,

02:13:28.133 --> 02:13:30.302
YOU KNOW, SOME OF THE SOME OF

02:13:30.369 --> 02:13:34.172
THE SAFETY ISSUES AROUND THAT.

02:13:34.239 --> 02:13:36.742
SO WHEN, WHEN DO YOU GET TO

02:13:36.808 --> 02:13:38.510
KIND OF TAKE ACTION TO

02:13:38.577 --> 02:13:40.312
INVESTIGATE, TO DO REPORTS AND

02:13:40.379 --> 02:13:41.880
HOW, HOW DOES THAT ALL WORK

02:13:41.947 --> 02:13:46.485
RIGHT NOW? WE MAINTAIN FRIENDLY

02:13:46.551 --> 02:13:47.519
RELATIONSHIPS WITH THE LAB. SO

02:13:47.586 --> 02:13:49.354
WE'RE GENERALLY IN TOUCH WITH

02:13:49.421 --> 02:13:51.556
THEM. WE CAN MAKE PROPOSALS, WE

02:13:51.623 --> 02:13:53.658
CAN TAKE PROPOSALS. BUT FOR ANY

02:13:53.725 --> 02:13:55.026
WORK TO HAPPEN, THERE HAS TO BE

02:13:55.093 --> 02:13:56.862
A SPECIFIC AGREEMENT BETWEEN

02:13:56.928 --> 02:13:58.029
META AND THE LAB OUTLINING THE

02:13:58.096 --> 02:13:59.731
SCOPE OF THAT AGREEMENT OF THE

02:13:59.798 --> 02:14:03.869
WORK. OKAY. AND HOW HOW OFTEN

02:14:03.935 --> 02:14:06.571
DO YOU DO INVESTIGATIONS AND

02:14:06.638 --> 02:14:08.039
HOW MANY STAFF DO YOU HAVE TO

02:14:08.106 --> 02:14:10.542
BE ABLE TO DO THIS WORK? AS OF

02:14:10.609 --> 02:14:12.411
TODAY, METER HAS ABOUT 50

02:14:12.477 --> 02:14:14.646
PEOPLE. WE ARE GROWING. YOU CAN

02:14:14.713 --> 02:14:16.181
SEE ALL OF OUR RELEASED WORK ON

02:14:16.248 --> 02:14:20.352
OUR WEBSITE. OKAY. THANK YOU.

02:14:20.419 --> 02:14:22.721
AS I MENTIONED, METER IS NOW IN

02:14:22.788 --> 02:14:24.656
A VOLUNTARY AGREEMENT TO BE

02:14:24.723 --> 02:14:26.691
EMBEDDED. AND IF THAT'S THE

02:14:26.758 --> 02:14:27.859
RIGHT WORD BUT EMBEDDED IN

02:14:27.926 --> 02:14:31.797
ANTHROPIC TO DO RESEARCH. YES.

02:14:31.863 --> 02:14:33.598
NOT INCIDENT RELATED, JUST

02:14:33.665 --> 02:14:35.867
RESEARCH. YOU MENTIONED THAT

02:14:35.934 --> 02:14:37.035
ONE OF THE THINGS THAT WAS

02:14:37.102 --> 02:14:38.804
IMPORTANT TO YOU WAS SUFFICIENT

02:14:38.870 --> 02:14:41.139
ACCESS. SO THE ACCESS YOU HAVE

02:14:41.206 --> 02:14:43.375
UNDER THAT PROPOSAL, AS

02:14:43.442 --> 02:14:45.177
ANTHROPIC HAS LAID IT OUT, IS

02:14:45.243 --> 02:14:47.679
THAT THAT METER HAS COMPANY

02:14:47.746 --> 02:14:48.780
LAPTOPS, OFFICE SPACE, HARDWARE

02:14:48.847 --> 02:14:50.549
AND INTERNAL TOOLS THAT ARE

02:14:50.615 --> 02:14:52.317
ROUGHLY EQUIVALENT TO THE

02:14:52.384 --> 02:14:55.954
EMPLOYEES. IT EXCLUDES CUSTOMER

02:14:56.021 --> 02:14:57.756
DATA, ATTORNEY CLIENT PRIVILEGE,

02:14:57.823 --> 02:14:59.558
AND CERTAIN SENSITIVE MODEL

02:14:59.624 --> 02:15:03.528
ARCHITECTURE. DETAILS. YES. DO

02:15:03.595 --> 02:15:04.963
YOU THINK THAT QUALIFIES AS

02:15:05.030 --> 02:15:07.132
SUFFICIENT ACCESS, OR ARE THERE

02:15:07.199 --> 02:15:14.806
THINGS MISSING IN THAT? I THINK

02:15:14.873 --> 02:15:16.374
I'D HAVE TO GET BACK TO YOU ON

02:15:16.441 --> 02:15:17.609
THAT. OKAY. WE WOULD APPRECIATE

02:15:17.676 --> 02:15:19.411
YOU TO GET BACK TO US. YEAH.

02:15:19.478 --> 02:15:21.246
THANK YOU. I MEAN, IT SOUNDS

02:15:21.313 --> 02:15:22.380
PRETTY EXTENSIVE, BUT I DON'T

02:15:22.447 --> 02:15:23.482
KNOW WHAT MODEL ARCHITECTURE

02:15:23.548 --> 02:15:24.483
DETAILS MEAN. I'M NOT DIRECTLY

02:15:24.549 --> 02:15:25.584
INVOLVED WITH THAT PROJECT. AND

02:15:25.650 --> 02:15:27.385
SO I HAVEN'T ASKED THE PEOPLE

02:15:27.452 --> 02:15:28.320
WHETHER THEIR ACCESS IS

02:15:28.386 --> 02:15:29.421
SUFFICIENT. OKAY. THAT WAS FROM

02:15:29.488 --> 02:15:30.522
ANTHROPIC OAN RELEASE AROUND

02:15:30.589 --> 02:15:32.324
IT AND IT SOUNDED GOOD. BUT AS

02:15:32.390 --> 02:15:33.492
AN ATTORNEY, I ALSO KNOW HOW

02:15:33.558 --> 02:15:35.293
MUCH CAN BE DEEMED ATTORNEY

02:15:35.360 --> 02:15:37.729
CLIENT PRIVILEGE. SO CURIOUS.

02:15:37.796 --> 02:15:39.731
ALSO, I THINK THAT PIGGYBACKING

02:15:39.798 --> 02:15:41.266
ON ASSEMBLYMEMBER PELLERIN'S

02:15:41.333 --> 02:15:43.468
QUESTION ABOUT, YOU KNOW, WHEN

02:15:43.535 --> 02:15:45.003
SHOULD COMPANIES REPORT. ONE OF

02:15:45.070 --> 02:15:46.705
THE THINGS THAT HAS BEEN REALLY,

02:15:46.771 --> 02:15:48.139
AS I MENTIONED EARLIER,

02:15:48.206 --> 02:15:51.009
FASCINATING TO ME IS THE

02:15:51.076 --> 02:15:52.444
FAILURE OF THE LABS TO

02:15:52.511 --> 02:15:54.246
UNDERSTAND THE AGENTIC ACTIONS,

02:15:54.312 --> 02:15:55.680
RIGHT, WHICH WE SAW IN THE

02:15:55.747 --> 02:15:58.984
HUGGING FACE INCIDENT. AND THEN

02:15:59.050 --> 02:16:00.151
AT THE POINT AT WHICH THEY

02:16:00.218 --> 02:16:01.953
REALIZE IT, BE IT THAT HUGGING

02:16:02.020 --> 02:16:03.989
FACE NOTIFIED THEM OR HOWEVER

02:16:04.055 --> 02:16:06.525
THEY IDENTIFY, I THINK IN THE

02:16:06.591 --> 02:16:08.627
THE INCIDENT WHERE ANTHROPIC'S

02:16:08.693 --> 02:16:09.761
CLAUDE WENT AND FILLED OUT THE

02:16:09.828 --> 02:16:12.264
POLICE REPORT IN PENNSYLVANIA,

02:16:12.330 --> 02:16:13.431
THEY, THROUGH INTERNAL AUDITS,

02:16:13.498 --> 02:16:15.133
I BELIEVE, UNDERSTOOD THAT TO

02:16:15.200 --> 02:16:19.471
BE THE CASE. IN THE CASE OF

02:16:19.538 --> 02:16:20.472
ANTHROPIC IN THE POLICE

02:16:20.539 --> 02:16:20.906
DEPARTMENT. IT'S MY

02:16:20.972 --> 02:16:21.706
UNDERSTANDING THAT THERE WAS

02:16:21.773 --> 02:16:25.176
ABOUT TEN DAYS BETWEEN THE

02:16:25.243 --> 02:16:26.211
REALIZATION THAT THE POLICE

02:16:26.278 --> 02:16:28.079
REPORT HAD BEEN FILED BY CLAUDE

02:16:28.146 --> 02:16:29.915
AND THE NOTIFICATION TO THE

02:16:29.981 --> 02:16:33.151
POLICE DEPARTMENT. I WILL SAY

02:16:33.218 --> 02:16:35.287
THAT I THINK THIS WAS A BENIGN.

02:16:35.353 --> 02:16:36.388
AS I UNDERSTAND IT, THERE

02:16:36.454 --> 02:16:37.522
WASN'T MUCH HARM THAT CAME FROM

02:16:37.589 --> 02:16:39.357
THAT, BUT I COULD SEE HOW

02:16:39.424 --> 02:16:40.425
SOMETHING AS SIMPLE AS THAT

02:16:40.492 --> 02:16:42.093
COULD DIVERT POLICE RESOURCES,

02:16:42.160 --> 02:16:43.228
SOMETHING THAT WE OBVIOUSLY

02:16:43.295 --> 02:16:44.262
BELIEVE ARE CRITICALLY

02:16:44.329 --> 02:16:45.697
IMPORTANT TO THE SAFETY OF OUR

02:16:45.764 --> 02:16:47.299
COMMUNITIES. AND FRANKLY, TEN

02:16:47.365 --> 02:16:48.867
DAYS SEEMS LIKE A LONG TIME. SO

02:16:48.934 --> 02:16:50.101
I JUST I'M NOT ASKING A

02:16:50.168 --> 02:16:51.870
QUESTION THERE, AND I KNOW YOU

02:16:51.937 --> 02:16:53.071
DON'T HAVE AN OPINION ON IT,

02:16:53.138 --> 02:16:54.973
BUT I DO THINK THAT WHEN YOU

02:16:55.040 --> 02:16:56.007
BROUGHT THAT UP, IT WAS

02:16:56.074 --> 02:16:57.876
SOMETHING TO NOTE THAT I THINK

02:16:57.943 --> 02:16:59.811
IT IS A QUESTION WE SHOULD BE

02:16:59.878 --> 02:17:00.879
ASKING IS WHEN IS THE

02:17:00.946 --> 02:17:01.947
APPROPRIATE TIME? AND MAYBE IT

02:17:02.013 --> 02:17:03.782
DOES DEPEND ON THE NATURE OF

02:17:03.848 --> 02:17:04.950
WHAT IS DISCOVERED, HOW QUICKLY

02:17:05.016 --> 02:17:06.785
IT NEEDS TO BE TOLD. BUT SINCE

02:17:06.851 --> 02:17:08.119
THIS IS AN SB 53 REVIEW HEARING,

02:17:08.186 --> 02:17:09.888
IT'S IMPORTANT TO NOTE THAT THE

02:17:09.955 --> 02:17:11.056
TIMELINES IN SB 53 RELATE ONLY

02:17:11.122 --> 02:17:13.091
TO WHERE 50 PEOPLE WILL DIE AT

02:17:13.158 --> 02:17:17.729
ONCE, OR THERE IS $1 BILLION Ó

02:17:17.796 --> 02:17:21.333
500 MILLION IN DAMAGE, AND I

02:17:21.399 --> 02:17:24.302
BELIEVE IT'S 70. IT IT'S 15

02:17:24.369 --> 02:17:25.704
DAYS. I WAS TRYING TO GET 72

02:17:25.770 --> 02:17:26.972
HOURS. SEE, I REMEMBER WHAT WAS

02:17:27.038 --> 02:17:28.640
NEGOTIATED. SO IT'S 15 DAYS.

02:17:28.707 --> 02:17:29.808
YOU'RE GOING TO KILL 50 PEOPLE

02:17:29.874 --> 02:17:31.743
AT ONCE, AND YOU HAVE 15 DAYS

02:17:31.810 --> 02:17:34.713
TO LET US KNOW THAT. I GUESS I

02:17:34.779 --> 02:17:36.915
AM OF THE PERSONAL OPINION THAT

02:17:36.982 --> 02:17:38.483
THAT MIGHT BE TOO LONG IF

02:17:38.550 --> 02:17:40.819
YOU'RE GOING TO KILL 50 PEOPLE.

02:17:40.885 --> 02:17:41.953
FRANKLY, I THINK IT'S TOO LONG

02:17:42.020 --> 02:17:43.822
IF YOU'RE GOING TO KILL ONE

02:17:43.888 --> 02:17:45.857
PERSON. SO I DO THINK THESE ARE

02:17:45.924 --> 02:17:47.259
QUESTIONS THAT WERE IMPORTANT

02:17:47.325 --> 02:17:49.561
TO ASK, AND WE DEFINITELY NEED

02:17:49.628 --> 02:17:53.865
TO REVISIT. SO I THINK, I MEAN,

02:17:53.932 --> 02:17:55.066
A LOT OF QUESTIONS HAVE BEEN

02:17:55.133 --> 02:17:56.835
ASKED. I WANT TO THANK YOU. I

02:17:56.901 --> 02:17:58.036
THINK IT'S A REALLY IMPORTANT

02:17:58.103 --> 02:18:01.006
PIECE OF THIS CONVERSATION. YOU

02:18:01.072 --> 02:18:02.374
KNOW, INDEPENDENT ASSESSORS, I

02:18:02.440 --> 02:18:04.242
REALLY LIKED THE CONVERSATION

02:18:04.309 --> 02:18:05.443
AROUND HOW DO WE GET ENOUGH

02:18:05.510 --> 02:18:07.479
PEOPLE TRAINED UP TO DO THIS,

02:18:07.545 --> 02:18:09.948
BOTH AT THE INDEPENDENT

02:18:10.015 --> 02:18:11.082
ENTITIES, FRANKLY, AT

02:18:11.149 --> 02:18:12.217
GOVERNMENT TOO. I MEAN, WHAT IS

02:18:12.283 --> 02:18:14.119
OUR ROLE HERE AND HOW DO WE

02:18:14.185 --> 02:18:15.053
BUILD INFRASTRUCTURE NECESSARY

02:18:15.120 --> 02:18:17.055
TO DO THE JOB WE WANT TO DO?

02:18:17.122 --> 02:18:18.289
AND I THINK WE ARE NOT THERE

02:18:18.356 --> 02:18:20.258
YET. AND SO I DO THINK IT IS

02:18:20.325 --> 02:18:22.060
IMPORTANT THAT WE TALK ABOUT

02:18:22.127 --> 02:18:23.228
THAT AND SUPPORT THE CREATION

02:18:23.295 --> 02:18:24.329
OF THIS MARKET, ALTHOUGH I DO

02:18:24.396 --> 02:18:26.197
THINK THAT THE WORK WE DID,

02:18:26.264 --> 02:18:27.232
INCLUDING MY BILL ON

02:18:27.298 --> 02:18:28.366
INDEPENDENT AUDITORS, WILL SPUR,

02:18:28.433 --> 02:18:30.268
AND WE ARE SEEING IT SPUR THAT

02:18:30.335 --> 02:18:31.403
ECONOMY AND START TO SEE THE

02:18:31.469 --> 02:18:33.304
GROWTH OF THAT INDUSTRY IN WAYS

02:18:33.371 --> 02:18:34.339
THAT IS REALLY HEALTHY AND

02:18:34.406 --> 02:18:36.975
HELPFUL. SO THANK YOU FOR THAT.

02:18:37.042 --> 02:18:38.410
I THINK THAT'S ALL THE

02:18:38.476 --> 02:18:39.477
QUESTIONS FOR YOU. SO I REALLY

02:18:39.544 --> 02:18:40.879
APPRECIATE YOU BEING HERE.

02:18:40.945 --> 02:18:42.147
THANK YOU SO MUCH FOR HAVING ME.

02:18:42.213 --> 02:18:43.948
I REALLY APPRECIATE IT. TO THE

02:18:44.015 --> 02:18:48.887
NEXT PANEL. SO THE NEXT PANEL

02:18:48.953 --> 02:18:51.356
IS OUR FRIENDLY CALIFORNIA

02:18:51.423 --> 02:18:54.325
COMPANIES. WE HAVE MORGAN DWYER,

02:18:54.392 --> 02:18:56.027
HEAD OF POLICY DEVELOPMENT AND

02:18:56.094 --> 02:18:57.896
OPERATIONS AT OPENAI. WE HAVE

02:18:57.962 --> 02:18:59.264
DAVID ORR, THE HEAD OF

02:18:59.330 --> 02:19:00.432
SAFEGUARDS AND ANTHROPIC, AND

02:19:00.498 --> 02:19:01.599
DOCTOR ALICE FRIEND, DIRECTOR

02:19:01.666 --> 02:19:03.568
OF GENERATIVE AI POLICY AT

02:19:03.635 --> 02:19:07.205
GOOGLE. AGAIN, BECAUSE I WAS SO

02:19:07.272 --> 02:19:08.306
SOUR ABOUT THAT ARTICLE

02:19:08.373 --> 02:19:09.708
YESTERDAY, I WILL NOTE THAT ON

02:19:09.774 --> 02:19:12.777
OUR AI PANEL OF COMPANY EXPERTS,

02:19:12.844 --> 02:19:15.547
WE HAVE MAJORITY WOMEN. SO

02:19:15.613 --> 02:19:19.851
THANK YOU ALL FOR BEING HERE.

02:19:19.918 --> 02:19:22.320
AND I DON'T KNOW, I MEAN, YOU

02:19:22.387 --> 02:19:24.055
COULD JUST GO IN THE ORDER THAT

02:19:24.122 --> 02:19:25.957
I JUST SAID IT. I DON'T KNOW

02:19:26.024 --> 02:19:28.126
THAT IT REALLY UNLESS WE HAVE

02:19:28.193 --> 02:19:30.195
SOMETHING TEED UP, I DON'T KNOW.

02:19:30.261 --> 02:19:32.097
MORGAN, DO YOU WANT TO GO FIRST?

02:19:32.163 --> 02:19:35.233
PERFECT. MORGAN. YES, I THINK

02:19:35.300 --> 02:19:37.736
WE HAVE SLIDES. IF MY SLIDES

02:19:37.802 --> 02:19:45.343
ARE TEED UP, I THINK I NEED.

02:19:45.410 --> 02:19:47.145
WE'RE ALL WATCHING THE SCREEN.

02:19:47.212 --> 02:19:53.985
THERE THEY ARE. OKAY, PERFECT.

02:19:54.052 --> 02:19:55.620
CHAIR BAUER-KAHAN AND MEMBERS

02:19:55.687 --> 02:19:57.155
OF THE COMMITTEE, THANK YOU FOR

02:19:57.222 --> 02:19:59.424
THE OPPORTUNITY TO PARTICIPATE

02:19:59.491 --> 02:20:01.926
TODAY. MY NAME IS DOCTOR MORGAN

02:20:01.993 --> 02:20:03.394
DWYER AND I LEAD OPENAI'S

02:20:03.461 --> 02:20:05.130
POLICY DEVELOPMENT AND

02:20:05.196 --> 02:20:06.998
OPERATIONS TEAM. I AM A

02:20:07.065 --> 02:20:08.433
SCIENTIST AND ENGINEER BY

02:20:08.500 --> 02:20:09.567
TRAINING, BUT I SPENT MOST OF

02:20:09.634 --> 02:20:11.669
MY CAREER IN PUBLIC SERVICE,

02:20:11.736 --> 02:20:12.804
INCLUDING AT THE WHITE HOUSE,

02:20:12.871 --> 02:20:14.539
THE PENTAGON AND THE DEPARTMENT

02:20:14.606 --> 02:20:17.442
OF COMMERCE. I SHARE YOUR

02:20:17.509 --> 02:20:18.843
COMMITMENT TO ENSURING THAT

02:20:18.910 --> 02:20:20.645
TECHNOLOGY IS DEVELOPED SAFELY

02:20:20.712 --> 02:20:22.447
AND ITS BENEFITS ARE WIDELY

02:20:22.514 --> 02:20:24.215
SHARED, A COMMITMENT THAT IS

02:20:24.282 --> 02:20:26.384
ALSO AT THE HEART OF OPENAI'S

02:20:26.451 --> 02:20:32.190
MISSION. I WORKED. OPENAI IS

02:20:32.257 --> 02:20:33.224
HAPPY TO WORK WITH THE

02:20:33.291 --> 02:20:35.059
CALIFORNIA LEGISLATURE. WE HAVE

02:20:35.126 --> 02:20:36.561
TESTIFIED HERE NUMEROUS TIMES

02:20:36.628 --> 02:20:38.663
OVER THE PAST YEAR, AND WE'VE

02:20:38.730 --> 02:20:40.064
BRIEFED MULTIPLE OFFICIALS

02:20:40.131 --> 02:20:43.001
ABOUT OUR EMERGING CAPABILITIES

02:20:43.067 --> 02:20:45.436
AND ABOUT OUR RECENT INCIDENTS

02:20:45.503 --> 02:20:46.905
TODAY, I'M HAPPY TO BE HERE TO

02:20:46.971 --> 02:20:49.073
SAY THAT OPENAI STRONGLY

02:20:49.140 --> 02:20:51.242
SUPPORTS SB 53 AND

02:20:51.309 --> 02:20:52.343
STRENGTHENING IT AS WE LEARN

02:20:52.410 --> 02:20:55.213
FROM REAL INCIDENTS. TODAY, I'M

02:20:55.280 --> 02:20:57.015
GOING TO EXPLAIN WHAT WE'VE

02:20:57.081 --> 02:20:59.150
LEARNED. AND FOR PRACTICAL

02:20:59.217 --> 02:21:02.053
UPDATES TO SB 53 BASED ON THOSE

02:21:02.120 --> 02:21:03.888
LEARNINGS, MONITORING FOR

02:21:03.955 --> 02:21:05.957
UNSAFE BEHAVIOR, MAINTAINING

02:21:06.024 --> 02:21:08.126
EFFECTIVE SHUTDOWN CONTROLS,

02:21:08.193 --> 02:21:09.594
STRENGTHENING INCIDENT

02:21:09.661 --> 02:21:11.796
REPORTING, AND ESTABLISHING

02:21:11.863 --> 02:21:13.231
INDEPENDENT TECHNICAL

02:21:13.298 --> 02:21:16.835
ASSESSMENTS. WE ARE IN A POLICY

02:21:16.901 --> 02:21:19.737
WINDOW. AI IS MOVING QUICKLY,

02:21:19.804 --> 02:21:21.906
AND CALIFORNIA HAS AN

02:21:21.973 --> 02:21:23.608
OPPORTUNITY TO LEAD IN ITS

02:21:23.675 --> 02:21:25.076
RESPONSE BY PUTTING PRACTICAL

02:21:25.143 --> 02:21:27.245
SAFEGUARDS IN PLACE BEFORE

02:21:27.312 --> 02:21:28.646
CAPABILITIES OUTPACE THE

02:21:28.713 --> 02:21:30.148
INSTITUTIONS RESPONSIBLE FOR

02:21:30.215 --> 02:21:32.450
GOVERNING THEM. OPENAI BELIEVES

02:21:32.517 --> 02:21:34.085
THAT THE US FEDERAL GOVERNMENT

02:21:34.152 --> 02:21:35.520
SHOULD LEAD ON NATIONAL

02:21:35.587 --> 02:21:36.955
SECURITY QUESTIONS THROUGH

02:21:37.021 --> 02:21:39.090
MANDATORY CAPABILITY BASED

02:21:39.157 --> 02:21:40.959
REQUIREMENTS. BUT UNTIL

02:21:41.025 --> 02:21:43.061
CONGRESS ACTS, STATES SHOULD

02:21:43.127 --> 02:21:45.630
CONTINUE TO MOVE, RAISE THE BAR

02:21:45.697 --> 02:21:47.432
AND ESTABLISH APPROACHES THAT

02:21:47.498 --> 02:21:49.934
CAN INFORM A DURABLE NATIONAL

02:21:50.001 --> 02:21:52.070
FRAMEWORK. THAT IS WHY WE

02:21:52.136 --> 02:21:54.606
STRONGLY SUPPORT SB 53. WE HAVE

02:21:54.672 --> 02:21:55.640
ACTIVELY COMPLIED WITH ITS

02:21:55.707 --> 02:21:57.842
APPLICABLE REQUIREMENTS. WE

02:21:57.909 --> 02:21:59.677
MAINTAIN A FRONTIER GOVERNANCE

02:21:59.744 --> 02:22:01.446
FRAMEWORK, PUBLISHED SYSTEM AND

02:22:01.512 --> 02:22:03.948
INTERNAL USE RISK ASSESSMENTS,

02:22:04.015 --> 02:22:06.518
AND MAINTAIN AN AI SAFETY AND

02:22:06.584 --> 02:22:08.253
INCIDENT RESPONSE PLAN. AND WE

02:22:08.319 --> 02:22:10.088
ALSO PROVIDE EMPLOYEES WITH

02:22:10.154 --> 02:22:12.290
PROTECTED CHANNELS TO RAISE

02:22:12.357 --> 02:22:14.592
CONCERNS. WE RECENTLY SUBMITTED

02:22:14.659 --> 02:22:16.561
OUR QUARTERLY REPORT TO CAL OES

02:22:16.628 --> 02:22:18.763
ON CATASTROPHIC RISK FROM

02:22:18.830 --> 02:22:20.164
INTERNAL DEPLOYMENT OF OUR

02:22:20.231 --> 02:22:21.966
MODELS, AND WE ALSO PUBLISHED A

02:22:22.033 --> 02:22:23.501
VERSION OF THAT REPORT ONLINE

02:22:23.568 --> 02:22:27.405
FOR INCREASED TRANSPARENCY.

02:22:27.472 --> 02:22:29.207
OPENAI ALSO SUPPORTS HARMONIZED

02:22:29.274 --> 02:22:30.642
LEGISLATION IN OTHER STATES

02:22:30.708 --> 02:22:32.377
THAT CAN ESTABLISH A NATIONAL

02:22:32.443 --> 02:22:34.178
BASELINE FOR CONGRESS TO BUILD

02:22:34.245 --> 02:22:36.714
UPON. BUT HARMONIZATION DOES

02:22:36.781 --> 02:22:38.750
NOT MEAN FREEZING REQUIREMENTS

02:22:38.816 --> 02:22:41.386
IN PLACE AS CAPABILITIES AND

02:22:41.452 --> 02:22:43.922
RISKS CHANGE, SAFEGUARDS MUST

02:22:43.988 --> 02:22:48.559
CHANGE AS WELL. RECENT

02:22:48.626 --> 02:22:50.228
INCIDENTS INVOLVING UNEXPECTED

02:22:50.295 --> 02:22:51.896
MODEL BEHAVIOR, INCLUDING THE

02:22:51.963 --> 02:22:53.665
HUGGING FACE INCIDENT,

02:22:53.731 --> 02:22:54.766
ILLUSTRATE THAT WE STILL HAVE

02:22:54.832 --> 02:22:56.167
MUCH TO LEARN ABOUT FRONTIER

02:22:56.234 --> 02:22:57.902
SAFETY AND THAT THERE IS ROOM

02:22:57.969 --> 02:23:00.471
FOR REGULATION TO RESPOND AS

02:23:00.538 --> 02:23:01.973
LESSONS TO THOSE LESSONS AS

02:23:02.040 --> 02:23:04.809
THEY EMERGE DURING INTERNAL

02:23:04.876 --> 02:23:06.945
CYBER EVALUATIONS. IN JULY,

02:23:07.011 --> 02:23:09.113
OPENAI MODELS CIRCUMVENTED

02:23:09.180 --> 02:23:10.348
CONTROLS INTENDED TO ISOLATE

02:23:10.415 --> 02:23:12.317
THEM FROM THE INTERNET AND

02:23:12.383 --> 02:23:14.118
ACCESS PARTS OF HUGGING FACE'S

02:23:14.185 --> 02:23:16.688
SYSTEMS WITHOUT AUTHORIZATION.

02:23:16.754 --> 02:23:18.856
THE INCIDENT WAS SERIOUS, AND

02:23:18.923 --> 02:23:20.992
WE TOOK CONCRETE STEPS IN

02:23:21.059 --> 02:23:23.161
RESPONSE. WE STOPPED

02:23:23.227 --> 02:23:24.362
EVALUATIONS, RESTRICTED ACCESS

02:23:24.429 --> 02:23:25.997
TO THE RESEARCH MODEL THAT WAS

02:23:26.064 --> 02:23:27.598
PRINCIPALLY RESPONSIBLE, AND

02:23:27.665 --> 02:23:28.900
WORKED WITH HUGGING FACE AND

02:23:28.967 --> 02:23:31.436
OUTSIDE SECURITY EXPERTS TO

02:23:31.502 --> 02:23:33.271
INVESTIGATE. WE PUBLISHED A

02:23:33.338 --> 02:23:35.039
TECHNICAL REPORT AND METER AND

02:23:35.106 --> 02:23:36.841
REDWOOD RESEARCH INDEPENDENTLY

02:23:36.908 --> 02:23:38.643
EXAMINED THE MODEL BEHAVIOR AND

02:23:38.710 --> 02:23:41.512
PUBLISHED THEIR FINDINGS. WE

02:23:41.579 --> 02:23:43.081
ALSO BOLSTERED OUR MONITORING

02:23:43.147 --> 02:23:45.483
AND SECURITY INFRASTRUCTURE,

02:23:45.550 --> 02:23:46.985
EXPANDED BARRIERS THAT LIMIT

02:23:47.051 --> 02:23:48.686
WHAT AI SYSTEMS CAN ACCESS

02:23:48.753 --> 02:23:50.588
DURING TRAINING, AND WE

02:23:50.655 --> 02:23:51.956
STRENGTHENED OUR INCIDENT

02:23:52.023 --> 02:23:54.826
RESPONSE PRACTICES. WE ARE

02:23:54.892 --> 02:23:56.294
CONTINUING TO INVESTIGATE

02:23:56.361 --> 02:23:58.363
UNEXPECTED MODEL BEHAVIOR,

02:23:58.429 --> 02:24:00.231
LEARN FROM PAST INCIDENTS, AND

02:24:00.298 --> 02:24:02.367
PUBLISH OUR FINDINGS. WE

02:24:02.433 --> 02:24:03.835
BELIEVE THAT TRANSPARENCY WITH

02:24:03.901 --> 02:24:06.371
THE PUBLIC, POLICYMAKERS,

02:24:06.437 --> 02:24:07.438
RESEARCHERS, AND OTHER

02:24:07.505 --> 02:24:08.940
COMPANIES IS ESSENTIAL IF WE

02:24:09.007 --> 02:24:10.308
ARE ALL GOING TO RAISE THE

02:24:10.375 --> 02:24:13.578
SAFETY BAR TOGETHER. LESSONS

02:24:13.644 --> 02:24:15.780
FROM OUR RECENT INCIDENTS

02:24:15.847 --> 02:24:17.548
INFORM FOR PRACTICAL UPDATES TO

02:24:17.615 --> 02:24:20.184
SB 53 THAT WE RECENTLY OUTLINED

02:24:20.251 --> 02:24:21.519
IN OUR LETTER TO GOVERNOR

02:24:21.586 --> 02:24:24.022
NEWSOM, AND THAT I'M GOING TO

02:24:24.088 --> 02:24:28.326
DISCUSS TODAY. SO FIRST, WE

02:24:28.393 --> 02:24:29.794
BELIEVE POLICYMAKERS SHOULD

02:24:29.861 --> 02:24:32.263
CONSIDER AMENDING SB 53 TO

02:24:32.330 --> 02:24:34.465
REQUIRE DEVELOPERS TO MONITOR

02:24:34.532 --> 02:24:36.567
UNSAFE MODEL BEHAVIOR DURING

02:24:36.634 --> 02:24:38.469
DEVELOPMENT AND EVALUATION.

02:24:38.536 --> 02:24:40.138
THAT MONITORING SHOULD LOOK FOR

02:24:40.204 --> 02:24:41.272
ATTEMPTS TO CIRCUMVENT THE

02:24:41.339 --> 02:24:43.074
DEVELOPERS OWN SAFEGUARDS OR

02:24:43.141 --> 02:24:44.876
ANOTHER ORGANIZATION SECURITY

02:24:44.942 --> 02:24:47.045
CONTROLS WITHOUT AUTHORIZATION,

02:24:47.111 --> 02:24:48.880
INCLUDING ATTEMPTS TO PROTECT

02:24:48.946 --> 02:24:50.581
OR TO ACCESS PROTECTED

02:24:50.648 --> 02:24:52.383
CONFIDENTIAL INFORMATION. IT

02:24:52.450 --> 02:24:53.851
SHOULD BE CONNECTED TO CLEAR

02:24:53.918 --> 02:24:55.286
ESCALATION PROCEDURES,

02:24:55.353 --> 02:24:56.754
INCLUDING WHO RECEIVES AN ALERT,

02:24:56.821 --> 02:24:58.589
WHO CAN INTERVENE WHEN WORK

02:24:58.656 --> 02:25:00.691
MUST PAUSE, AND WHAT MUST BE

02:25:00.758 --> 02:25:05.730
ADDRESSED BEFORE WORK RESUMES.

02:25:05.797 --> 02:25:07.932
SECOND, POLICYMAKERS

02:25:07.999 --> 02:25:09.000
UNDERSTANDABLY WANT CONFIDENCE

02:25:09.067 --> 02:25:10.802
THAT THERE IS A RELIABLE WAY TO

02:25:10.868 --> 02:25:13.071
STOP AN UNSAFE SYSTEM, A GOAL

02:25:13.137 --> 02:25:14.138
THAT'S OFTEN DESCRIBED AS A

02:25:14.205 --> 02:25:16.541
KILL SWITCH. WE SUPPORT THAT

02:25:16.607 --> 02:25:19.077
GOAL, BUT THAT TERM CAN IMPLY

02:25:19.143 --> 02:25:20.878
THAT A SINGLE SWITCH AT THE

02:25:20.945 --> 02:25:21.913
ORIGINATING DEVELOPER WILL

02:25:21.979 --> 02:25:23.714
ALWAYS BE ABLE TO STOP A MODEL

02:25:23.781 --> 02:25:26.584
WHEREVER IT IS RUNNING. IN

02:25:26.651 --> 02:25:28.786
PRACTICE, STOPPING UNSAFE AI

02:25:28.853 --> 02:25:31.322
OPERATIONS REQUIRES SEVERAL

02:25:31.389 --> 02:25:32.857
CONTROLS AND COORDINATED ACTION

02:25:32.924 --> 02:25:35.660
BY DIFFERENT ACTORS. FOR THAT

02:25:35.726 --> 02:25:36.727
REASON, WE PREFER THE TERM

02:25:36.794 --> 02:25:38.896
SHUTDOWN CONTROLS. DEPENDING ON

02:25:38.963 --> 02:25:40.998
THE SYSTEM AND RISK. THOSE

02:25:41.065 --> 02:25:42.834
CONTROLS MAY MEAN PAUSING A

02:25:42.900 --> 02:25:45.736
TRAINING RUN, ISOLATING AN

02:25:45.803 --> 02:25:48.272
AGENT, REVOKING TOOL OR NETWORK

02:25:48.339 --> 02:25:51.075
ACCESS, OR SUSPENDING A SERVICE.

02:25:51.142 --> 02:25:52.910
FRONTIER DEVELOPERS LIKE OPENAI

02:25:52.977 --> 02:25:54.712
SHOULD BE REQUIRED TO MAINTAIN

02:25:54.779 --> 02:25:56.914
EFFECTIVE SHUTDOWN CONTROLS FOR

02:25:56.981 --> 02:25:58.316
OPERATIONS WITHIN THEIR CONTROL

02:25:58.382 --> 02:26:00.485
AND REQUIRED TO USE THEM WHEN

02:26:00.551 --> 02:26:02.653
NEEDED TO STOP UNSAFE

02:26:02.720 --> 02:26:04.422
OPERATIONS. THEIR SAFETY

02:26:04.489 --> 02:26:05.857
FRAMEWORK SHOULD EXPLAIN HOW

02:26:05.923 --> 02:26:08.392
MONITORING LEADS TO ESCALATION,

02:26:08.459 --> 02:26:10.094
INTERVENTION, INVESTIGATION,

02:26:10.161 --> 02:26:11.996
AND SAFE RESUMPTION OF AFFECTED

02:26:12.063 --> 02:26:15.800
WORK. THIRD, POLICYMAKERS

02:26:15.867 --> 02:26:18.169
SHOULD CONSIDER AMENDING SB 53

02:26:18.236 --> 02:26:19.971
TO STRENGTHEN REPORTING AND

02:26:20.037 --> 02:26:21.305
NOTICE FOR SERIOUS CYBER

02:26:21.372 --> 02:26:23.508
INCIDENTS. IT SHOULD REQUIRE

02:26:23.574 --> 02:26:24.976
PROMPT WRITTEN NOTICE TO AN

02:26:25.042 --> 02:26:26.777
AFFECTED ORGANIZATION. WHEN A

02:26:26.844 --> 02:26:28.212
FRONTIER MODEL EXPLOITS A

02:26:28.279 --> 02:26:30.314
SECURITY VULNERABILITY AND

02:26:30.381 --> 02:26:32.483
MATERIALLY ASSESSES, ALTERS OR

02:26:32.550 --> 02:26:34.785
DESTROYS PROTECTED SYSTEMS OR

02:26:34.852 --> 02:26:36.821
CONFIDENTIAL INFORMATION, IT

02:26:36.888 --> 02:26:38.289
SHOULD ALSO EXPAND CRITICAL

02:26:38.356 --> 02:26:40.491
INCIDENT REPORTING TO COVER

02:26:40.558 --> 02:26:42.260
SERIOUS LOSS OF CONTROL

02:26:42.326 --> 02:26:43.694
INCIDENTS THAT CAUSE MEASURABLE

02:26:43.761 --> 02:26:45.429
ECONOMIC HARM OR DISRUPTION TO

02:26:45.496 --> 02:26:47.298
THIRD PARTY SYSTEMS UNDER

02:26:47.365 --> 02:26:50.134
CLEARLY DEFINED CONDITIONS. WE

02:26:50.201 --> 02:26:52.003
ALSO BELIEVE THAT VOLUNTARY

02:26:52.069 --> 02:26:54.138
REPORTING SHOULD CONTINUE SO

02:26:54.205 --> 02:26:56.674
THAT POLICY MAKERS, THE PUBLIC,

02:26:56.741 --> 02:26:58.442
RESEARCHERS AND INDUSTRY CAN

02:26:58.509 --> 02:26:59.877
LEARN FROM MISALIGNMENT

02:26:59.944 --> 02:27:01.679
INCIDENTS THAT DO NOT MEET A

02:27:01.746 --> 02:27:06.350
MANDATORY THRESHOLD. FINALLY,

02:27:06.417 --> 02:27:08.152
WE SUPPORT REQUIRING

02:27:08.219 --> 02:27:09.620
INDEPENDENT TECHNICAL

02:27:09.687 --> 02:27:11.088
ASSESSMENTS TO GIVE

02:27:11.155 --> 02:27:12.123
POLICYMAKERS INSIGHTS INTO

02:27:12.190 --> 02:27:14.659
RISKS FROM HIGHLY CAPABLE

02:27:14.725 --> 02:27:16.794
INTERNAL DEPLOYMENTS AND THE

02:27:16.861 --> 02:27:17.895
EFFECTIVENESS OF THEIR

02:27:17.962 --> 02:27:20.064
SAFEGUARDS. INITIAL ASSESSMENTS

02:27:20.131 --> 02:27:21.866
SHOULD FOCUS ON ALIGNMENT,

02:27:21.933 --> 02:27:23.601
CONTAINMENT AND MONITORING

02:27:23.668 --> 02:27:25.836
DURING FRONTIER TRAINING,

02:27:25.903 --> 02:27:27.305
EVALUATION AND INTERNAL

02:27:27.371 --> 02:27:29.040
DEPLOYMENT, INCLUDING HOW

02:27:29.106 --> 02:27:30.875
AUTOMATED AI RESEARCH MAY

02:27:30.942 --> 02:27:33.711
INTENSIFY THOSE CHALLENGES.

02:27:33.778 --> 02:27:36.214
EXCUSE ME, STANDARDS FOR THESE

02:27:36.280 --> 02:27:38.015
ASSESSMENTS DON'T YET EXIST,

02:27:38.082 --> 02:27:39.817
AND AS WE HEARD EARLIER, FEW

02:27:39.884 --> 02:27:40.952
ORGANIZATIONS HAVE THE

02:27:41.018 --> 02:27:42.753
NECESSARY CAPACITY TO DO THIS

02:27:42.820 --> 02:27:45.990
WORK. THAT'S WHY WE PREFER OR

02:27:46.057 --> 02:27:47.525
THAT'S WHY A FRAMEWORK CONSISTS.

02:27:47.592 --> 02:27:49.594
WE PREFER A FEDERAL FRAMEWORK

02:27:49.660 --> 02:27:53.231
WITH. FOR THIS REQUIREMENT,

02:27:53.297 --> 02:27:55.299
CONSISTENT WITH HIGH LEVELS OF

02:27:55.366 --> 02:27:57.134
SECURITY, ACCESS AND ACCESS TO

02:27:57.201 --> 02:27:59.303
SENSITIVE INFORMATION. BUT

02:27:59.370 --> 02:28:01.405
UNTIL CONGRESS ACTS TO ENACT

02:28:01.472 --> 02:28:03.207
THAT FEDERAL FRAMEWORK, WE

02:28:03.274 --> 02:28:05.109
BELIEVE THAT CALIFORNIA CAN

02:28:05.176 --> 02:28:09.714
BUILD ON SB 813, AB 1405 AND

02:28:09.780 --> 02:28:12.950
UPDATES TO SB 53 TO DEVELOP

02:28:13.017 --> 02:28:14.385
STANDARDS AND PAVE THE WAY FOR

02:28:14.452 --> 02:28:16.921
THAT FEDERAL FRAMEWORK. ONE

02:28:16.988 --> 02:28:19.123
PLACE TO BEGIN IS WITH SAFETY

02:28:19.190 --> 02:28:20.925
CASES. SAFETY CASE APPROACHES

02:28:20.992 --> 02:28:22.693
ARE USED IN AVIATION AND IN

02:28:22.760 --> 02:28:25.196
NUCLEAR POWER. A SAFETY CASE

02:28:25.263 --> 02:28:26.998
EXPLAINS WHY A DEVELOPER

02:28:27.064 --> 02:28:28.432
BELIEVES SERIOUS RISKS ARE

02:28:28.499 --> 02:28:29.867
ADEQUATELY, ADEQUATELY MANAGED

02:28:29.934 --> 02:28:32.403
FOR TRAINING, EVALUATION OR

02:28:32.470 --> 02:28:33.838
DEPLOYMENT. IT CONNECTS THAT

02:28:33.904 --> 02:28:35.640
JUDGMENT TO EVIDENCE AND

02:28:35.706 --> 02:28:37.475
IDENTIFIES THE ASSUMPTIONS,

02:28:37.541 --> 02:28:38.909
UNCERTAINTIES AND REMAINING

02:28:38.976 --> 02:28:41.746
RISKS. QUALIFIED INDEPENDENT

02:28:41.812 --> 02:28:43.214
ASSESSORS SHOULD EXAMINE THAT

02:28:43.281 --> 02:28:45.049
EVIDENCE, METHODS AND REASONING

02:28:45.116 --> 02:28:47.151
SUPPORTING THOSE SAFETY CASES,

02:28:47.218 --> 02:28:49.320
INCLUDING MODEL EVALUATIONS AND

02:28:49.387 --> 02:28:51.188
SAFEGUARD EFFECTIVENESS. THEY

02:28:51.255 --> 02:28:52.890
SHOULD TEST ASSUMPTIONS,

02:28:52.957 --> 02:28:54.759
IDENTIFY GAPS AND EXPLAIN

02:28:54.825 --> 02:28:55.760
UNCERTAINTIES OR LIMITS TO

02:28:55.826 --> 02:28:58.696
THEIR ACCESS. TO DO THIS WORK

02:28:58.763 --> 02:29:00.531
WELL, INDEPENDENT ASSESSORS

02:29:00.598 --> 02:29:02.266
WILL NEED ACCESS TO RELEVANT

02:29:02.333 --> 02:29:05.202
EVIDENCE SYSTEMS AND PERSONNEL,

02:29:05.269 --> 02:29:06.604
BALANCED WITH PROTECTIONS FOR

02:29:06.671 --> 02:29:08.072
SENSITIVE RESEARCH,

02:29:08.139 --> 02:29:10.274
INTELLECTUAL PROPERTY

02:29:10.341 --> 02:29:12.376
INFRASTRUCTURE, SECURITY,

02:29:12.443 --> 02:29:13.844
CONFIDENTIALITY AND CONFLICT OF

02:29:13.911 --> 02:29:17.248
INTEREST SAFEGUARDS. CALIFORNIA

02:29:17.315 --> 02:29:18.849
CAN HELP BUILD THIS ECOSYSTEM

02:29:18.916 --> 02:29:20.785
BY CONVENING TECHNICAL EXPERTS

02:29:20.851 --> 02:29:22.820
AND STANDARDS BODIES,

02:29:22.887 --> 02:29:23.821
COORDINATING WITH THE US

02:29:23.888 --> 02:29:25.623
FEDERAL GOVERNMENT ON SHARED

02:29:25.690 --> 02:29:28.192
CRITERIA, INVESTING IN ASSESSOR

02:29:28.259 --> 02:29:29.994
CAPACITY, AND BUILDING ITS OWN

02:29:30.061 --> 02:29:32.530
TECHNICAL OVERSIGHT STAFF.

02:29:32.596 --> 02:29:33.698
BUILDING THAT PUBLIC CAPACITY

02:29:33.764 --> 02:29:35.766
IS IMPORTANT BECAUSE GOVERNMENT

02:29:35.833 --> 02:29:38.602
MUST RETAIN RESPONSIBILITY FOR

02:29:38.669 --> 02:29:40.805
STANDARDS, ACCREDITATION AND

02:29:40.871 --> 02:29:41.906
CONSEQUENTIAL DECISIONS ABOUT

02:29:41.972 --> 02:29:44.375
SAFETY AND PEACE. INDEPENDENT

02:29:44.442 --> 02:29:45.509
TECHNICAL ASSESSMENTS SHOULD

02:29:45.576 --> 02:29:46.043
PROVIDE EVIDENCE TO INFORM

02:29:46.110 --> 02:29:47.945
THOSE DECISIONS, NOT MAKE THEM

02:29:48.012 --> 02:29:50.381
IN THE PLACE OF DEMOCRATICALLY

02:29:50.448 --> 02:29:54.752
ACCOUNTABLE INSTITUTIONS. IN

02:29:54.819 --> 02:29:55.820
CONCLUSION, AI IS MOVING

02:29:55.886 --> 02:29:58.356
QUICKLY AND CALIFORNIA HAS AN

02:29:58.422 --> 02:30:00.091
OPPORTUNITY TO LEAD IN THE

02:30:00.157 --> 02:30:02.259
RESPONSE. AS I SAID AT THE

02:30:02.326 --> 02:30:03.394
BEGINNING, THE POLICY WINDOW IS

02:30:03.461 --> 02:30:05.930
OPEN. CALIFORNIA CAN BUILD ON

02:30:05.996 --> 02:30:08.899
SB 53 STRONG FOUNDATION, APPLY

02:30:08.966 --> 02:30:10.901
THE LESSONS FROM REAL INCIDENTS,

02:30:10.968 --> 02:30:12.770
AND ESTABLISH SAFEGUARDS THAT

02:30:12.837 --> 02:30:14.538
REMAIN PRACTICAL AS THE

02:30:14.605 --> 02:30:16.674
TECHNOLOGY EVOLVES. WE ARE

02:30:16.741 --> 02:30:18.142
COMMITTED TO WORKING WITH THE

02:30:18.209 --> 02:30:19.276
COMMITTEE AND WITH THE STATE TO

02:30:19.343 --> 02:30:21.345
DO JUST THAT. THANK YOU, AND I

02:30:21.412 --> 02:30:23.547
LOOK FORWARD TO YOUR QUESTIONS.

02:30:23.614 --> 02:30:24.715
THANK YOU SO MUCH. I JUST WANT

02:30:24.782 --> 02:30:26.684
TO, BEFORE WE MOVE ON, WHICH WE

02:30:26.751 --> 02:30:27.718
WILL. ONE SECOND, MY

02:30:27.785 --> 02:30:28.786
UNDERSTANDING IN OUR HEARING IN

02:30:28.853 --> 02:30:31.455
EARLY AUGUST WAS THAT NONE OF

02:30:31.522 --> 02:30:34.358
OPENAI'S MODELS HAD MET THE

02:30:34.425 --> 02:30:36.994
THRESHOLD UNDER SB 53. WE ASKED

02:30:37.061 --> 02:30:38.295
THIS QUESTION IN OUR JOINT

02:30:38.362 --> 02:30:39.697
HEARING, AND WE WERE TOLD THAT

02:30:39.764 --> 02:30:41.565
ONLY ONE OF THE LABS HAD

02:30:41.632 --> 02:30:43.000
REACHED THE THRESHOLD OF TEN TO

02:30:43.067 --> 02:30:45.236
THE 26 FLOP AND THE $500

02:30:45.302 --> 02:30:46.270
MILLION REVENUE. I THINK YOU

02:30:46.337 --> 02:30:47.371
ALL HAVE $500 MILLION IN

02:30:47.438 --> 02:30:49.173
REVENUE. SO I ASSUMED IT WAS

02:30:49.240 --> 02:30:51.108
THE FLOP. IS IT NOW YOU'RE DID

02:30:51.175 --> 02:30:52.376
I HEAR YOU SAY THAT YOU HAVE,

02:30:52.443 --> 02:30:54.211
AT LEAST ON SOME MODELS, REACH

02:30:54.278 --> 02:30:56.180
THE TEN TO THE 26 FLOP AND ARE

02:30:56.247 --> 02:30:58.849
COMPLYING WITH SB 53. SO THE

02:30:58.916 --> 02:31:00.251
SPECIFICS OF COMPUTE THRESHOLDS

02:31:00.317 --> 02:31:01.719
THAT'S KEPT CONFIDENTIAL. SO

02:31:01.786 --> 02:31:03.320
IT'S NOT SOMETHING I CAN TALK

02:31:03.387 --> 02:31:04.522
ABOUT TODAY. WHAT YOU HEARD ME

02:31:04.588 --> 02:31:06.323
SAY IS WE DID SUBMIT AN

02:31:06.390 --> 02:31:08.959
INTERNAL RISK REPORT TO CAL OES

02:31:09.026 --> 02:31:10.361
RECENTLY. AND I THINK MORE

02:31:10.428 --> 02:31:11.762
IMPORTANTLY, REGARDLESS OF

02:31:11.829 --> 02:31:14.698
COMPUTE THRESHOLDS, WE MADE OUR

02:31:14.765 --> 02:31:17.134
THE INCIDENTS PUBLIC. WE. WHEN

02:31:17.201 --> 02:31:18.803
WE LEARNED OF OUR HUGGING FACE

02:31:18.869 --> 02:31:20.070
AND OTHER INCIDENTS, WE. WE

02:31:20.137 --> 02:31:22.573
MADE OUR FINDINGS OF THOSE

02:31:22.640 --> 02:31:23.808
INCIDENTS PUBLIC AND WE INVITED

02:31:23.874 --> 02:31:25.409
THIRD PARTIES TO INVESTIGATE

02:31:25.476 --> 02:31:26.877
THEM AND MAKE THE INCIDENTS

02:31:26.944 --> 02:31:28.546
PUBLIC AS WELL. I APPRECIATE

02:31:28.612 --> 02:31:30.848
THAT, I THINK THAT AND MAYBE

02:31:30.915 --> 02:31:32.550
YOU WANT TO DO IT OUTSIDE OF AN

02:31:32.616 --> 02:31:33.717
OPEN HEARING, BUT I THINK ONE

02:31:33.784 --> 02:31:36.287
OF THE QUESTIONS WE ARE

02:31:36.353 --> 02:31:37.688
WRESTLING WITH IS THAT THE

02:31:37.755 --> 02:31:38.889
COMPUTE THRESHOLD, SB 53, IS

02:31:38.956 --> 02:31:40.724
TEN TO THE 26TH. IN THE EU,

02:31:40.791 --> 02:31:43.060
IT'S TEN TO THE 25TH, AS YOU

02:31:43.127 --> 02:31:47.064
KNOW, AND IT WAS OUR

02:31:47.131 --> 02:31:47.731
UNDERSTANDING THAT MANY OF THE

02:31:47.798 --> 02:31:49.433
MODELS THAT WE BELIEVE, BASED

02:31:49.500 --> 02:31:50.668
ON THE EXPERTS WE'VE TALKED TO

02:31:50.734 --> 02:31:52.570
THAT ARE CAPABLE OF CAUSING

02:31:52.636 --> 02:31:53.537
SIGNIFICANT HARM, ARE WELL

02:31:53.604 --> 02:31:54.772
BELOW THE THRESHOLD OF TEN TO

02:31:54.839 --> 02:31:56.807
THE 26. AND SO THIS IS GOING TO

02:31:56.874 --> 02:31:58.676
BE SOMETHING WE TALK ABOUT. AND

02:31:58.742 --> 02:31:59.877
I THINK THAT YOU HEARD CLEARLY

02:31:59.944 --> 02:32:01.645
FROM MEMBERS UP AND DOWN THE

02:32:01.712 --> 02:32:02.813
STATE IN OUR OPENING REMARKS,

02:32:02.880 --> 02:32:03.747
THAT VOLUNTARY WILL NOT BE

02:32:03.814 --> 02:32:04.849
SUFFICIENT FOR THE STATE OF

02:32:04.915 --> 02:32:05.916
CALIFORNIA, THAT WE BELIEVE

02:32:05.983 --> 02:32:07.618
THAT OUR COMMUNITY DESERVE MORE.

02:32:07.685 --> 02:32:08.886
AND IT IS OUR ROLE TO CREATE

02:32:08.953 --> 02:32:10.621
STANDARDS. AND SO IT'S A

02:32:10.688 --> 02:32:11.689
CONVERSATION THAT I THINK WE

02:32:11.755 --> 02:32:12.890
WILL HAVE TO HAVE. BUT THANK

02:32:12.957 --> 02:32:14.658
YOU. WE DO APPRECIATE THE

02:32:14.725 --> 02:32:15.726
VOLUNTARY RELEASE, BUT I THINK

02:32:15.793 --> 02:32:17.628
WE NEED TO EXPECT MORE. NOW, I

02:32:17.695 --> 02:32:18.762
DON'T I SHOULD LOOK AT MY

02:32:18.829 --> 02:32:19.930
AGENDA. ARE YOU NEXT DOCTOR

02:32:19.997 --> 02:32:24.201
FRIEND OR. OKAY, SURE. ALL

02:32:24.268 --> 02:32:27.071
RIGHT. CHAIR BAUER K HÌNH,

02:32:27.137 --> 02:32:28.405
MEMBERS OF THE COMMITTEE AND

02:32:28.472 --> 02:32:30.274
ASSEMBLY MEMBERS JOINING US

02:32:30.341 --> 02:32:31.709
TODAY. THANK YOU FOR THE

02:32:31.775 --> 02:32:33.477
OPPORTUNITY TO SPEAK WITH YOU.

02:32:33.544 --> 02:32:35.246
MY NAME IS ALICE FRIEND. I

02:32:35.312 --> 02:32:36.780
SERVE AS DIRECTOR FOR

02:32:36.847 --> 02:32:38.148
GENERATIVE AI POLICY AT GOOGLE.

02:32:38.215 --> 02:32:40.384
I ALSO HAVE A NATIONAL SECURITY

02:32:40.451 --> 02:32:42.486
BACKGROUND. GOOGLE WAS DON'T

02:32:42.553 --> 02:32:43.787
MEAN TO INTERRUPT YOU, DOCTOR,

02:32:43.854 --> 02:32:45.022
BUT DID YOU HAVE SLIDES? I DO

02:32:45.089 --> 02:32:46.824
NOT. OKAY, PERFECT. I'M GOING

02:32:46.891 --> 02:32:47.958
TO TALK. I'M SORRY, I KNOW I

02:32:48.025 --> 02:32:49.860
NEVER DO SLIDES, SO I'M WITH

02:32:49.927 --> 02:32:50.895
YOU. HAVING WORKED AT THE

02:32:50.961 --> 02:32:52.796
PENTAGON, I HAVE AN AVERSION TO

02:32:52.863 --> 02:32:55.099
SLIDES. ACTUALLY. GOOGLE WAS

02:32:55.165 --> 02:32:57.635
FOUNDED IN CALIFORNIA OVER 28

02:32:57.701 --> 02:32:59.370
YEARS AGO TO ORGANIZE THE

02:32:59.436 --> 02:33:01.605
WORLD'S INFORMATION AND MAKE IT

02:33:01.672 --> 02:33:03.340
UNIVERSALLY ACCESSIBLE AND

02:33:03.407 --> 02:33:05.543
USEFUL. OUR GOAL HAS ALWAYS

02:33:05.609 --> 02:33:07.711
BEEN TO IMPROVE THE LIVES OF AS

02:33:07.778 --> 02:33:10.247
MANY PEOPLE AS POSSIBLE THROUGH

02:33:10.314 --> 02:33:12.750
TECHNOLOGY. HERE IN CALIFORNIA,

02:33:12.816 --> 02:33:15.619
OUR AI TOOLS ARE HELPING DETECT

02:33:15.686 --> 02:33:18.122
WILDFIRES WITH SATELLITE IMAGES

02:33:18.188 --> 02:33:19.957
PROVIDING EARLY WARNINGS FOR

02:33:20.024 --> 02:33:22.059
EARTHQUAKES AND FLOODS, AND

02:33:22.126 --> 02:33:24.295
REDUCING FARM WATER USE WHILE

02:33:24.361 --> 02:33:27.131
INCREASING CROP YIELDS. AI IS

02:33:27.197 --> 02:33:29.667
MAKING IT POSSIBLE TO SOLVE

02:33:29.733 --> 02:33:31.468
ECONOMIC, SCIENTIFIC, AND

02:33:31.535 --> 02:33:33.604
SOCIAL PROBLEMS FASTER THAN

02:33:33.671 --> 02:33:36.440
EVER. BUT WE KNOW IT POSES

02:33:36.507 --> 02:33:39.043
CHALLENGES AS WELL. TO ADDRESS

02:33:39.109 --> 02:33:41.145
THESE CHALLENGES, WE TAKE A

02:33:41.211 --> 02:33:43.314
BOLD BUT RESPONSIBLE APPROACH

02:33:43.380 --> 02:33:46.951
TO AI, EMPHASIZING CUTTING EDGE

02:33:47.017 --> 02:33:48.719
RESEARCH TO GUIDE OUR DECISION

02:33:48.786 --> 02:33:50.788
MAKING AND INFORM DEVELOPMENT

02:33:50.854 --> 02:33:54.525
OF OUR AI MODELS AND SYSTEMS.

02:33:54.592 --> 02:33:55.960
TODAY, I'D LIKE TO SHARE

02:33:56.026 --> 02:33:58.796
INFORMATION ABOUT THREE TOPICS

02:33:58.862 --> 02:34:00.598
A SET OF INCIDENTS INVOLVING

02:34:00.664 --> 02:34:03.167
GOOGLE'S GEMINI SYSTEM,

02:34:03.233 --> 02:34:05.703
GOOGLE'S AI SAFETY AND SECURITY

02:34:05.769 --> 02:34:07.805
APPROACH, AND KEY

02:34:07.871 --> 02:34:09.573
CONSIDERATIONS FOR FRONTIER AI

02:34:09.640 --> 02:34:12.576
POLICY AND REGULATION. I WANT

02:34:12.643 --> 02:34:14.979
TO START BY ADDRESSING A SET OF

02:34:15.045 --> 02:34:16.814
TESTING INCIDENTS INVOLVING

02:34:16.880 --> 02:34:19.083
GOOGLE'S GEMINI SYSTEM. HERE'S

02:34:19.149 --> 02:34:22.219
WHAT HAPPENED DURING STANDARD

02:34:22.286 --> 02:34:24.455
CYBER BENCHMARK EVALUATIONS RUN

02:34:24.521 --> 02:34:27.591
BY AN EXTERNAL EVALUATOR,

02:34:27.658 --> 02:34:30.894
GEMINI ACCESSED THE INTERNET.

02:34:30.961 --> 02:34:32.663
IMPORTANTLY, GEMINI DID NOT

02:34:32.730 --> 02:34:35.199
HACK OR DISABLE SAFEGUARDS TO

02:34:35.265 --> 02:34:37.267
GAIN ACCESS TO THE INTERNET.

02:34:37.334 --> 02:34:39.103
ACCESS WAS MADE AVAILABLE BY

02:34:39.169 --> 02:34:42.773
THE EVALUATOR. THE SYSTEM FOUND

02:34:42.840 --> 02:34:45.275
PUBLIC INFORMATION ONLINE AND

02:34:45.342 --> 02:34:47.077
OBTAINED OR GUESSED CREDENTIALS

02:34:47.144 --> 02:34:49.980
TO ACCESS WEBSITES OF THREE

02:34:50.047 --> 02:34:52.449
SEPARATE ENTITIES. IN EACH OF

02:34:52.516 --> 02:34:55.319
THE THREE INCIDENTS, GEMINI

02:34:55.386 --> 02:34:59.256
STOPPED BEFORE CAUSING HARM.

02:34:59.323 --> 02:35:01.792
THE IMPACTED ENTITIES AND U.S.

02:35:01.859 --> 02:35:03.193
FEDERAL AUTHORITIES WERE

02:35:03.260 --> 02:35:04.428
NOTIFIED, AND WE WORKED WITH

02:35:04.495 --> 02:35:06.497
THE EVALUATOR ON CHANGES TO

02:35:06.563 --> 02:35:09.333
THEIR TESTING PROCESSES. THESE

02:35:09.400 --> 02:35:10.901
THREE INCIDENTS UNDERSCORE THAT

02:35:10.968 --> 02:35:13.270
AS FRONTIER MODELS BECOME MORE

02:35:13.337 --> 02:35:16.206
CAPABLE, AI SAFETY REQUIRES

02:35:16.273 --> 02:35:19.043
ACTIVE, LAYERED DEFENSES NOT

02:35:19.109 --> 02:35:20.878
ONLY IN TESTING ENVIRONMENTS,

02:35:20.944 --> 02:35:24.081
BUT ALSO ACROSS EVERY STAGE OF

02:35:24.148 --> 02:35:27.351
DEVELOPMENT AND DEPLOYMENT. AT

02:35:27.418 --> 02:35:29.920
GOOGLE, WE EMPLOY A LAYERED

02:35:29.987 --> 02:35:31.722
DEFENSE IN DEPTH APPROACH

02:35:31.789 --> 02:35:33.457
THROUGHOUT THE AI SYSTEM

02:35:33.524 --> 02:35:35.659
LIFECYCLE, COVERING MODEL

02:35:35.726 --> 02:35:38.495
TRAINING AND TESTING, SYSTEM

02:35:38.562 --> 02:35:40.364
DEPLOYMENT, AND POST-LAUNCH

02:35:40.431 --> 02:35:42.499
MONITORING. WE IDENTIFY AND

02:35:42.566 --> 02:35:44.301
ASSESS RISKS THROUGH A BROAD

02:35:44.368 --> 02:35:46.437
RANGE OF SOURCES, WHICH INCLUDE

02:35:46.503 --> 02:35:48.939
SAFETY RESEARCH, INPUT FROM

02:35:49.006 --> 02:35:51.175
EXTERNAL EXPERTS, AND

02:35:51.241 --> 02:35:54.044
ADVERSARIAL TESTING. WE APPLY

02:35:54.111 --> 02:35:55.479
SAFETY FINE TUNING TO THE MODEL

02:35:55.546 --> 02:35:57.247
AS PART OF OUR ALIGNMENT

02:35:57.314 --> 02:35:59.016
APPROACH AND LAYER FILTERS AND

02:35:59.083 --> 02:36:01.218
OTHER CONTROLS OUTSIDE THE

02:36:01.285 --> 02:36:03.320
MODEL TO ADD DETERMINISTIC

02:36:03.387 --> 02:36:05.489
GUARDRAILS OVER PROBABILISTIC

02:36:05.556 --> 02:36:08.425
OUTPUTS BEFORE LAUNCH, MODELS

02:36:08.492 --> 02:36:10.227
AND SYSTEMS ARE EVALUATED

02:36:10.294 --> 02:36:13.030
AGAINST AGAINST BENCHMARKS FOR

02:36:13.097 --> 02:36:15.265
SAFETY, PRIVACY AND SECURITY,

02:36:15.332 --> 02:36:17.434
AND RISK MITIGATIONS AND ROBUST

02:36:17.501 --> 02:36:18.802
PROVENANCE SOLUTIONS ARE

02:36:18.869 --> 02:36:21.338
APPLIED. TO DO THIS WORK, WE

02:36:21.405 --> 02:36:23.574
FOLLOW A SET OF FRAMEWORKS,

02:36:23.640 --> 02:36:25.743
INCLUDING OUR FRONTIER SAFETY

02:36:25.809 --> 02:36:27.478
FRAMEWORK, OUR APPROACH TO

02:36:27.544 --> 02:36:29.613
TECHNICAL AGI SAFETY AND

02:36:29.680 --> 02:36:32.516
SECURITY, AND OUR AI CONTROL

02:36:32.583 --> 02:36:34.318
ROADMAP, WHICH ADDRESSES

02:36:34.384 --> 02:36:37.554
AGENTIC AI. SPECIFICALLY. THIS

02:36:37.621 --> 02:36:39.022
MONTH, GOOGLE ANNOUNCED OUR

02:36:39.089 --> 02:36:41.492
LATEST MODEL, GEMINI FOR

02:36:41.558 --> 02:36:43.293
ARGONNE, WHOSE DEVELOPMENT AND

02:36:43.360 --> 02:36:44.762
LAUNCH DEMONSTRATE HOW THIS

02:36:44.828 --> 02:36:47.631
WORKS IN PRACTICE. WHEN ROLLING

02:36:47.698 --> 02:36:49.867
OUT ARGONNE, WE USED A PHASED

02:36:49.933 --> 02:36:51.969
APPROACH. GIVEN ITS FRONTIER

02:36:52.035 --> 02:36:53.837
CAPABILITIES. WE ENGAGED IN THE

02:36:53.904 --> 02:36:55.973
US GOVERNMENT'S VOLUNTARY

02:36:56.039 --> 02:36:58.475
PROCESS FOR PRERELEASE MODEL

02:36:58.542 --> 02:37:00.277
ACCESS AND MADE ARGONNE AND

02:37:00.344 --> 02:37:02.780
CODE VENDOR, OUR SOFTWARE

02:37:02.846 --> 02:37:04.948
VULNERABILITY PATCHING TOOL

02:37:05.015 --> 02:37:06.683
AVAILABLE TO TRUSTED CYBER

02:37:06.750 --> 02:37:09.253
DEFENDERS THROUGH OUR FAIR WIND

02:37:09.319 --> 02:37:12.122
PROGRAM. FROM THE BEGINNING, WE

02:37:12.189 --> 02:37:13.957
TRAINED GEMINI FOR ARGONNE TO

02:37:14.024 --> 02:37:16.460
RESPECT SYSTEM BOUNDARIES,

02:37:16.527 --> 02:37:18.595
REPORT NEGATIVE FINDINGS

02:37:18.662 --> 02:37:20.731
HONESTLY, AND PAUSE TO ASK FOR

02:37:20.798 --> 02:37:24.001
HUMAN CONFIRMATION. AS AN ADDED

02:37:24.067 --> 02:37:25.803
DEFENSE, AN AUTOMATED MONITOR

02:37:25.869 --> 02:37:27.905
READS THE MODEL'S STEP BY STEP

02:37:27.971 --> 02:37:30.107
CHAIN OF THOUGHT AND ACTIONS IN

02:37:30.174 --> 02:37:33.377
REAL TIME. THIS ALLOWS VARIOUS

02:37:33.443 --> 02:37:34.411
INTERVENTIONS, SUCH AS FOR

02:37:34.478 --> 02:37:36.613
CASES TO BE FLAGGED FOR HUMAN

02:37:36.680 --> 02:37:39.449
REVIEW OR ACTIONS TO BE BLOCKED.

02:37:39.516 --> 02:37:41.285
THIS MONITOR RUNS IN OUR

02:37:41.351 --> 02:37:43.053
PRODUCTS ON OUR INTERNAL CODING

02:37:43.120 --> 02:37:44.521
AGENTS DURING TRAINING AND

02:37:44.588 --> 02:37:46.723
DURING OUR CAPABILITY

02:37:46.790 --> 02:37:49.159
EVALUATIONS. DURING DEVELOPMENT,

02:37:49.226 --> 02:37:50.961
WE ALSO ADOPTED SAFETY

02:37:51.028 --> 02:37:53.497
GUARDRAILS TO AVOID MISUSE AND

02:37:53.564 --> 02:37:55.299
RISKS OF MODEL MISALIGNMENT,

02:37:55.365 --> 02:37:56.800
WHICH CAN OCCUR WHEN AN AGENT

02:37:56.867 --> 02:37:58.035
PURSUES A GOAL DIFFERENT FROM

02:37:58.101 --> 02:38:00.470
THE USER'S INTENT. WE WRAPPED

02:38:00.537 --> 02:38:02.172
THE MODEL IN THREE LAYERS OF

02:38:02.239 --> 02:38:04.708
REAL TIME PROTECTION TO PREVENT

02:38:04.775 --> 02:38:06.944
ATTEMPTS TO JAILBREAK OR BYPASS

02:38:07.010 --> 02:38:09.012
OUR SAFEGUARDS WITH DANGEROUS

02:38:09.079 --> 02:38:11.181
REQUESTS. THESE INCLUDE

02:38:11.248 --> 02:38:12.649
SCREENING, WHERE SMALLER MODELS

02:38:12.716 --> 02:38:15.152
REVIEW INCOMING REQUESTS AND

02:38:15.219 --> 02:38:16.520
CONVERSATIONS FOR MALICIOUS OR

02:38:16.587 --> 02:38:19.456
DANGEROUS REQUESTS. CHECKS

02:38:19.523 --> 02:38:21.592
USING INTERNAL SIGNALS TO SPOT

02:38:21.658 --> 02:38:23.427
WHEN THE MODEL IS ACTIVATING

02:38:23.493 --> 02:38:25.362
KNOWLEDGE ABOUT CYBER ATTACKS

02:38:25.429 --> 02:38:28.465
OR BIOLOGICAL THREATS, AND

02:38:28.532 --> 02:38:29.933
CONTINUOUS MONITORING. WHERE WE

02:38:30.000 --> 02:38:31.702
COMBINE THESE SIGNALS ACROSS

02:38:31.768 --> 02:38:33.837
MULTIPLE RESPONSES AND TAKE

02:38:33.904 --> 02:38:35.672
ACTION IF THEY BECOME

02:38:35.739 --> 02:38:38.008
PROBLEMATIC. THIS COMBINATION

02:38:38.075 --> 02:38:39.610
OF TECHNIQUES HAS MADE OUR

02:38:39.676 --> 02:38:41.011
SYSTEMS BETTER AT PREVENTING

02:38:41.078 --> 02:38:43.914
REAL WORLD HARM. THIS IS JUST A

02:38:43.981 --> 02:38:45.382
PART OF OUR PROCESS FOR GEMINI

02:38:45.449 --> 02:38:47.017
FOR ARGONNE. MORE DETAILS HAVE

02:38:47.084 --> 02:38:48.285
BEEN PUBLISHED IN THE MODELS

02:38:48.352 --> 02:38:49.720
SAFETY AND SECURITY REPORT,

02:38:49.786 --> 02:38:50.954
WHICH WE ARE HAPPY TO FURNISH

02:38:51.021 --> 02:38:53.223
TO YOU AFTER THE HEARING AND IS

02:38:53.290 --> 02:38:54.424
AVAILABLE ON OUR WEBSITE. RIGHT

02:38:54.491 --> 02:38:56.894
NOW, WE ARE AT A PIVOTAL MOMENT

02:38:56.960 --> 02:38:58.962
IN AI DEVELOPMENT, WHICH IS WHY

02:38:59.029 --> 02:39:00.831
WE SHARE THE LEGISLATURE'S

02:39:00.898 --> 02:39:03.000
OBJECTIVE OF ENSURING THAT AI

02:39:03.066 --> 02:39:04.835
SYSTEMS DEPLOYED WITHIN

02:39:04.902 --> 02:39:07.704
CALIFORNIA AND AROUND THE WORLD

02:39:07.771 --> 02:39:10.173
EARN PUBLIC TRUST BY OPERATING

02:39:10.240 --> 02:39:12.376
UNDER HIGH STANDARDS OF SAFETY

02:39:12.442 --> 02:39:15.212
AND TRANSPARENCY. GOOGLE

02:39:15.279 --> 02:39:17.814
BELIEVES THAT TO ACHIEVE THIS,

02:39:17.881 --> 02:39:19.182
EFFECTIVE AND COHESIVE

02:39:19.249 --> 02:39:20.984
REGULATORY FRAMEWORKS WILL BE

02:39:21.051 --> 02:39:22.819
ANCHORED BY NATIONALLY AND

02:39:22.886 --> 02:39:24.588
INTERNATIONALLY RECOGNIZED

02:39:24.655 --> 02:39:26.423
STANDARDS, COMPLEMENTED BY

02:39:26.490 --> 02:39:28.926
HARMONIZED FEDERAL AND STATE

02:39:28.992 --> 02:39:30.694
REGULATIONS CONDUCIVE TO

02:39:30.761 --> 02:39:33.563
RESPONSIBLE INNOVATION. AS THE

02:39:33.630 --> 02:39:35.766
ASSEMBLY CONSIDERS FURTHER AI

02:39:35.832 --> 02:39:36.767
REGULATION, IT IS IMPORTANT

02:39:36.833 --> 02:39:39.336
THAT FRAMEWORKS PRIORITIZE AND

02:39:39.403 --> 02:39:41.872
PROMOTE HIGH VALUE AI SAFETY

02:39:41.939 --> 02:39:44.074
APPROACHES. DEVELOPERS SHOULD

02:39:44.141 --> 02:39:45.876
TAKE A DEFENSE IN DEPTH

02:39:45.943 --> 02:39:47.811
APPROACH, LAYERING SAFEGUARDS

02:39:47.878 --> 02:39:49.813
AT THE MODEL, SYSTEM AND

02:39:49.880 --> 02:39:51.982
ACCOUNT LEVELS SO THAT IF ONE

02:39:52.049 --> 02:39:54.084
FAILS, OTHERS CAN CATCH ISSUES

02:39:54.151 --> 02:39:56.987
BEFORE A USER IS IMPACTED.

02:39:57.054 --> 02:39:58.455
MODEL SAFETY ASSESSMENTS SHOULD

02:39:58.522 --> 02:40:00.223
BE BASED ON A HOLISTIC

02:40:00.290 --> 02:40:02.826
CONSIDERATION OF BOTH MODEL

02:40:02.893 --> 02:40:05.262
CAPABILITIES AND THE ABILITY OF

02:40:05.329 --> 02:40:06.797
MITIGATIONS, SAFEGUARDS AND

02:40:06.863 --> 02:40:08.865
SYSTEM CONTROLS TO ADDRESS

02:40:08.932 --> 02:40:11.802
IDENTIFIED RISKS, LAYERED

02:40:11.868 --> 02:40:13.203
SECURITY STRATEGIES SHOULD

02:40:13.270 --> 02:40:16.807
FEATURE EFFECTIVE, SECURE AND

02:40:16.873 --> 02:40:18.575
PROPORTIONATE SAFETY VALVE

02:40:18.642 --> 02:40:20.410
INTERVENTION SYSTEMS. SUCH

02:40:20.477 --> 02:40:21.845
SYSTEMS COULD ENCOMPASS

02:40:21.912 --> 02:40:23.246
MONITORING, SHOULD ENCOMPASS

02:40:23.313 --> 02:40:24.715
MONITORING OR DETECTION TO

02:40:24.781 --> 02:40:27.584
IDENTIFY ISSUES AS THEY OCCUR.

02:40:27.651 --> 02:40:28.685
ATTRIBUTION TO ISOLATE THE

02:40:28.752 --> 02:40:30.454
SOURCE OF THE ISSUE TO ALLOW

02:40:30.520 --> 02:40:32.990
TARGETED INTERVENTION AND

02:40:33.056 --> 02:40:34.825
GRADUATED RESPONSES THAT ARE

02:40:34.891 --> 02:40:36.259
PROPORTIONATE TO CONCERNING

02:40:36.326 --> 02:40:38.762
ACTIVITY. THIS IS SAFER AND

02:40:38.829 --> 02:40:40.597
MORE PRACTICAL THAN CENTRALIZED

02:40:40.664 --> 02:40:43.066
OFF SWITCHES, WHICH CREATE

02:40:43.133 --> 02:40:45.635
SINGLE POINTS OF FAILURE, ARE

02:40:45.702 --> 02:40:47.070
INEFFECTIVE FOR OPEN WEIGHT

02:40:47.137 --> 02:40:49.606
MODELS AND RISK UNNECESSARILY

02:40:49.673 --> 02:40:51.041
DISRUPTING DOWNSTREAM SYSTEMS

02:40:51.108 --> 02:40:53.176
FOR HEALTH CARE, FINANCE AND

02:40:53.243 --> 02:40:56.046
DEFENSIVE CYBER SECURITY. AS

02:40:56.113 --> 02:40:58.248
THE STATE SETS STANDARDS TO

02:40:58.315 --> 02:40:59.649
PROFESSIONALIZE EVALUATORS

02:40:59.716 --> 02:41:04.021
UNDER SB 813 AND AB 1405,

02:41:04.087 --> 02:41:05.455
BUILDING A TRUSTWORTHY

02:41:05.522 --> 02:41:07.024
INDEPENDENT EVALUATOR ECOSYSTEM

02:41:07.090 --> 02:41:09.359
WILL BE PARAMOUNT FOR SAFETY

02:41:09.426 --> 02:41:11.561
ASSURANCE, SECURITY AND

02:41:11.628 --> 02:41:13.397
CONTINUED INNOVATION IN

02:41:13.463 --> 02:41:15.465
CALIFORNIA. DRAWING ON OUR

02:41:15.532 --> 02:41:16.933
EXPERIENCE WITH THIRD PARTY

02:41:17.000 --> 02:41:20.203
EXPERTS AND EVALUATORS, WE ALSO

02:41:20.270 --> 02:41:21.638
RECOMMEND THAT POLICIES

02:41:21.705 --> 02:41:23.040
MAINTAIN THE ABILITY OF

02:41:23.106 --> 02:41:24.808
DEVELOPERS TO WORK WITH DOMAIN

02:41:24.875 --> 02:41:27.711
EXPERTS. EVALUATORS ARE USUALLY

02:41:27.778 --> 02:41:29.546
SPECIALISTS IN SPECIFIC RISKS

02:41:29.613 --> 02:41:32.082
AND DOMAINS SUCH AS

02:41:32.149 --> 02:41:34.251
MISALIGNMENT, CYBERSECURITY, OR

02:41:34.317 --> 02:41:36.720
BIOLOGY. A SINGLE EVALUATOR

02:41:36.787 --> 02:41:39.256
CANNOT TEST FOR ALL RISKS.

02:41:39.322 --> 02:41:40.624
THAT'S WHY FOR GEMINI, FOR

02:41:40.690 --> 02:41:42.125
ARGONNE, WE WORKED WITH U.S.

02:41:42.192 --> 02:41:45.395
AND U.K. GOVERNMENT EXPERTS, 35

02:41:45.462 --> 02:41:47.264
DOMAIN EXPERTS ACROSS FIELDS

02:41:47.330 --> 02:41:50.067
SUCH AS CHEMICAL, BIOLOGICAL,

02:41:50.133 --> 02:41:51.835
RADIOLOGICAL, AND NUCLEAR

02:41:51.902 --> 02:41:53.336
WEAPONIZATION, AND AT LEAST

02:41:53.403 --> 02:41:55.772
SEVEN INDEPENDENT EVALUATION

02:41:55.839 --> 02:41:58.141
ORGANIZATIONS. CYBERSECURITY

02:41:58.208 --> 02:41:59.509
AND OPERATIONAL SECURITY SHOULD

02:41:59.576 --> 02:42:01.578
BE NON-NEGOTIABLE ELEMENTS OF

02:42:01.645 --> 02:42:03.380
MODEL DEVELOPMENT AND

02:42:03.447 --> 02:42:05.549
DEPLOYMENT. GOOGLE'S FRONTIER

02:42:05.615 --> 02:42:07.684
ENVIRONMENTS STORE PROPRIETARY

02:42:07.751 --> 02:42:09.486
WEIGHTS, SOVEREIGN CLOUD

02:42:09.553 --> 02:42:10.887
ARCHITECTURES, AND SENSITIVE

02:42:10.954 --> 02:42:12.389
VULNERABILITY DATA PROTECTED BY

02:42:12.456 --> 02:42:14.858
STRICT SECURITY PROTOCOLS.

02:42:14.925 --> 02:42:16.660
GIVEN RECENT HACKING AND

02:42:16.726 --> 02:42:18.295
ESPIONAGE TARGETING DEVELOPERS

02:42:18.361 --> 02:42:21.364
AND EVALUATORS, THREATS TO AI,

02:42:21.431 --> 02:42:23.567
AI FIRMS, CUSTOMERS, PUBLIC

02:42:23.633 --> 02:42:25.702
SAFETY, AND NATIONAL SECURITY

02:42:25.769 --> 02:42:27.871
REQUIRE SERIOUS ATTENTION AND

02:42:27.938 --> 02:42:31.074
SUSTAINED VIGILANCE. FINALLY,

02:42:31.141 --> 02:42:32.742
DEVELOPERS AND POLICYMAKERS

02:42:32.809 --> 02:42:34.978
SHOULD MAINTAIN EVALUATORS

02:42:35.045 --> 02:42:36.813
INDEPENDENCE. EXISTING

02:42:36.880 --> 02:42:38.615
OVERSIGHT MODELS FOR SENSITIVE

02:42:38.682 --> 02:42:39.716
INFRASTRUCTURE REQUIRE

02:42:39.783 --> 02:42:41.151
EVALUATORS MAINTAIN AN ARM'S

02:42:41.218 --> 02:42:42.252
LENGTH RELATIONSHIP WITH

02:42:42.319 --> 02:42:44.387
ENTITIES THEY INSPECT. THIS

02:42:44.454 --> 02:42:45.655
AVOIDS CONFLICTS OF INTEREST

02:42:45.722 --> 02:42:47.090
AND MAINTAINS PUBLIC CONFIDENCE

02:42:47.157 --> 02:42:49.426
IN FINDINGS. THANK YOU AGAIN

02:42:49.493 --> 02:42:51.027
FOR THE OPPORTUNITY TO TESTIFY

02:42:51.094 --> 02:42:52.329
AT THIS IMPORTANT HEARING. I

02:42:52.395 --> 02:42:54.164
LOOK FORWARD TO YOUR QUESTIONS.

02:42:54.231 --> 02:42:55.232
THANK YOU, DOCTOR BRENNAN. JUST

02:42:55.298 --> 02:42:57.734
TO BE FAIR, I ACTUALLY DON'T

02:42:57.801 --> 02:42:59.903
KNOW IF GEMINI HAS COMPLYING

02:42:59.970 --> 02:43:02.506
WITH SB 53. DO YOU KNOW I HAVE

02:43:02.572 --> 02:43:04.407
TO CHECK ON THAT. OKAY. I MADE

02:43:04.474 --> 02:43:06.042
A NOTE WHEN YOU ASKED THE

02:43:06.109 --> 02:43:08.912
QUESTION EARLIER. I MEAN,

02:43:08.979 --> 02:43:10.780
OBVIOUSLY WE WILL COMPLY WITH

02:43:10.847 --> 02:43:11.982
WHEN YOU MEET THE THRESHOLD. OF

02:43:12.048 --> 02:43:13.517
COURSE, I DON'T HAVE I DON'T

02:43:13.583 --> 02:43:15.285
WANT TO TELL YOU THE WRONG

02:43:15.352 --> 02:43:17.220
THING, SO I'LL GET BACK TO YOU.

02:43:17.287 --> 02:43:23.994
THANK YOU. MR. CHAIR. ANY.

02:43:24.060 --> 02:43:26.096
CHAIR, MEMBERS OF THE COMMITTEE,

02:43:26.163 --> 02:43:27.964
THANK YOU FOR THE INVITATION.

02:43:28.031 --> 02:43:29.099
MEMBERS NOT OF THE COMMITTEE.

02:43:29.166 --> 02:43:30.834
NICE TO SEE YOU AS WELL. I ALSO

02:43:30.901 --> 02:43:32.302
AM ENJOYING A SLIDE FREE

02:43:32.369 --> 02:43:34.804
EXISTENCE HERE. MY NAME IS

02:43:34.871 --> 02:43:37.674
DAVID OR I AM HEAD OF

02:43:37.741 --> 02:43:39.910
SAFEGUARDS AT ANTHROPIC. MY

02:43:39.976 --> 02:43:41.645
TEAM IS ONE OF SEVERAL AT

02:43:41.711 --> 02:43:43.380
ANTHROPIC THAT WORKS ON SAFETY

02:43:43.446 --> 02:43:45.649
ALONGSIDE ALIGNMENT RESEARCH,

02:43:45.715 --> 02:43:46.683
INTERPRETABILITY, FRONTIER

02:43:46.750 --> 02:43:49.619
THREAT EVALUATIONS, AND OTHERS.

02:43:49.686 --> 02:43:50.987
MY TEAM'S WORK IS TO BUILD THE

02:43:51.054 --> 02:43:52.822
SYSTEMS THAT STOP OUR AI MODELS

02:43:52.889 --> 02:43:55.358
FROM BEING MISUSED OR DOING

02:43:55.425 --> 02:43:56.726
THINGS THAT THEY SHOULD NOT BE

02:43:56.793 --> 02:44:00.363
DOING. SO OUR SYSTEMS INCLUDE

02:44:00.430 --> 02:44:01.798
CLASSIFIERS THAT DETECT AND

02:44:01.865 --> 02:44:03.934
BLOCK HARMFUL REQUESTS AND

02:44:04.000 --> 02:44:06.136
ACTIONS, MONITORING SYSTEMS,

02:44:06.203 --> 02:44:09.005
RED TEAMING AND RESEARCH ON

02:44:09.072 --> 02:44:11.174
KEEPING AI AGENTS SAFE. WHEN I

02:44:11.241 --> 02:44:12.175
TALK ABOUT TESTING AND

02:44:12.242 --> 02:44:13.343
SAFEGUARDS TODAY, I'M

02:44:13.410 --> 02:44:14.044
DESCRIBING THE WORK THAT MY

02:44:14.110 --> 02:44:16.246
TEAM DOES ON A DAILY BASIS.

02:44:16.313 --> 02:44:17.614
TODAY I'LL TALK ABOUT THREE

02:44:17.681 --> 02:44:19.883
THINGS. FIRST AND FOREMOST,

02:44:19.950 --> 02:44:21.952
SAFETY IS CENTRAL TO HOW WE DO

02:44:22.018 --> 02:44:24.120
OUR WORK AT ANTHROPIC,

02:44:24.187 --> 02:44:25.589
INCLUDING THE INCIDENTS THAT WE

02:44:25.655 --> 02:44:26.823
FOUND AND DISCLOSED EARLIER

02:44:26.890 --> 02:44:29.926
THIS YEAR. SECOND, I'LL SPEND A

02:44:29.993 --> 02:44:31.294
MINUTE TALKING ABOUT THE SAFETY

02:44:31.361 --> 02:44:33.096
MEASURES THAT WE HAVE IN PLACE

02:44:33.163 --> 02:44:35.632
TODAY. AND THIRD, AS YOU KNOW,

02:44:35.699 --> 02:44:37.400
WE WERE THE ONLY FRONTIER LAB

02:44:37.467 --> 02:44:39.302
TO SUPPORT SB 53 BEFORE IT WAS

02:44:39.369 --> 02:44:41.104
SIGNED INTO LAW. AND I'LL

02:44:41.171 --> 02:44:42.339
DESCRIBE HOW WE THINK WE CAN

02:44:42.405 --> 02:44:43.940
WORK WITH YOU TO MAKE SB 53

02:44:44.007 --> 02:44:46.076
STRONGER. OKAY, SO LET ME TURN

02:44:46.142 --> 02:44:48.311
TO SAFETY. SAFETY IS CENTRAL TO

02:44:48.378 --> 02:44:50.747
HOW WE WORK. WE BELIEVE AI CAN

02:44:50.814 --> 02:44:53.283
DO ENORMOUS GOOD, INCLUDING TO

02:44:53.350 --> 02:44:55.118
DRIVE POSITIVE IMPACTS ACROSS

02:44:55.185 --> 02:44:56.486
HEALTHCARE, LIFE SCIENCES,

02:44:56.553 --> 02:44:57.988
SOFTWARE SECURITY, PUBLIC

02:44:58.054 --> 02:45:00.156
HEALTH, AND MORE. WE ALSO

02:45:00.223 --> 02:45:02.626
BELIEVE AI CARRIES REAL RISKS,

02:45:02.692 --> 02:45:05.195
AND GETTING THIS WRONG HAS REAL

02:45:05.262 --> 02:45:06.263
CONSEQUENCES FOR INDIVIDUALS,

02:45:06.329 --> 02:45:07.731
FOR INSTITUTIONS, AND FOR

02:45:07.797 --> 02:45:10.533
PUBLIC TRUST. BECAUSE BOTH THE

02:45:10.600 --> 02:45:12.335
BENEFITS AND THE RISKS ARE REAL.

02:45:12.402 --> 02:45:14.204
WE HAVE TO BUILD AI QUITE

02:45:14.271 --> 02:45:16.239
CAREFULLY. THAT MEANS WE PUT

02:45:16.306 --> 02:45:17.774
OUR MODELS THROUGH EXTENSIVE

02:45:17.841 --> 02:45:19.242
TESTING, AND WE GO OUT OF OUR

02:45:19.309 --> 02:45:20.910
WAY TO LOOK FOR PROBLEMS THAT

02:45:20.977 --> 02:45:22.012
MIGHT ONLY SHOW UP IN THE

02:45:22.078 --> 02:45:23.947
RAREST OF CIRCUMSTANCES. SOME

02:45:24.014 --> 02:45:25.315
OF THESE TESTS PUT MODELS IN

02:45:25.382 --> 02:45:26.416
SITUATIONS THAT ARE DESIGNED TO

02:45:26.483 --> 02:45:28.952
MIMIC REAL WORLD USE. AS MODELS

02:45:29.019 --> 02:45:30.387
GROW MORE CAPABLE AND

02:45:30.453 --> 02:45:31.454
AUTONOMOUS, THEY CAN PURSUE

02:45:31.521 --> 02:45:33.290
GOALS IN WAYS THAT NOBODY

02:45:33.356 --> 02:45:34.424
INTENDED. FOR THAT REASON, WE

02:45:34.491 --> 02:45:35.792
RUN EACH TEST HUNDREDS OR

02:45:35.859 --> 02:45:37.294
THOUSANDS OF TIMES. THAT WAY,

02:45:37.360 --> 02:45:38.862
IF THE MODEL DOES SOMETHING

02:45:38.928 --> 02:45:40.530
EXPECTED, EVEN ONE TIME, WE CAN

02:45:40.597 --> 02:45:42.165
SPOT IT. AND SOMETIMES THE

02:45:42.232 --> 02:45:43.233
PROBLEMS THESE TESTS TURN UP

02:45:43.300 --> 02:45:45.135
ISN'T IN THE MODEL, BUT IN THE

02:45:45.201 --> 02:45:46.936
DESIGN OF THE TEST ITSELF. IN

02:45:47.003 --> 02:45:48.371
JULY, WE REPORTED THREE

02:45:48.438 --> 02:45:49.472
INCIDENTS IN WHICH CLOUD MODELS

02:45:49.539 --> 02:45:50.940
GAINED UNAUTHORIZED ACCESS TO

02:45:51.007 --> 02:45:53.443
REAL COMPUTER SYSTEMS. IN

02:45:53.510 --> 02:45:54.544
SEPTEMBER, WE PUBLISHED A

02:45:54.611 --> 02:45:55.945
DETAILED ASSESSMENT OF THESE

02:45:56.012 --> 02:45:57.047
INCIDENTS, AS WELL AS ONE

02:45:57.113 --> 02:45:58.448
ADDITIONAL INCIDENT WE LATER

02:45:58.515 --> 02:45:59.549
IDENTIFIED AS PART OF OUR

02:45:59.616 --> 02:46:01.685
ONGOING REVIEW. ALL OF THESE

02:46:01.751 --> 02:46:03.153
INCIDENTS OCCURRED DURING

02:46:03.219 --> 02:46:04.220
CYBERSECURITY EVALUATIONS BUILT

02:46:04.287 --> 02:46:05.322
BY THE SAME THIRD PARTY

02:46:05.388 --> 02:46:07.457
EVALUATION PARTNER. CLOUD WAS

02:46:07.524 --> 02:46:09.092
TOLD THAT IT WAS IN A

02:46:09.159 --> 02:46:10.760
SIMULATION WITH NO INTERNET

02:46:10.827 --> 02:46:12.095
ACCESS. HOWEVER, DUE TO A

02:46:12.162 --> 02:46:13.196
MISCONFIGURATION IN THE TESTING

02:46:13.263 --> 02:46:14.097
ENVIRONMENT, IT WAS CONNECTED

02:46:14.164 --> 02:46:17.133
TO THE OPEN INTERNET. MORE

02:46:17.200 --> 02:46:18.234
RECENTLY, ON OCTOBER 9TH, WE

02:46:18.301 --> 02:46:19.936
PUBLISHED A REPORT ON CERTAIN

02:46:20.003 --> 02:46:21.137
UNINTENDED ACTIONS OUR MODELS

02:46:21.204 --> 02:46:22.605
TOOK ON EXTERNAL WEBSITES AND

02:46:22.672 --> 02:46:23.973
SYSTEMS DURING OUR EVALUATION,

02:46:24.040 --> 02:46:25.842
TESTING, AND INTERNAL USE. TO

02:46:25.909 --> 02:46:27.177
OUR KNOWLEDGE, NONE OF THE

02:46:27.243 --> 02:46:28.278
CASES WE HAVE DISCLOSED HAVE

02:46:28.345 --> 02:46:29.379
INVOLVED A COMPROMISE OF OUR

02:46:29.446 --> 02:46:31.214
INTERNAL SYSTEMS OR CUSTOMER

02:46:31.281 --> 02:46:32.282
DATA MAINTAINED ON THOSE

02:46:32.349 --> 02:46:34.851
SYSTEMS. TO BE CLEAR, OUR NORTH

02:46:34.918 --> 02:46:36.519
STAR IS BUILDING CLOUD SO THAT

02:46:36.586 --> 02:46:38.054
IT BEHAVES SAFELY EVEN WHEN

02:46:38.121 --> 02:46:39.856
OTHER SAFEGUARDS ARE MISSING OR

02:46:39.923 --> 02:46:42.425
FAIL. I ALSO WANT TO EMPHASIZE

02:46:42.492 --> 02:46:43.793
THAT WE'RE COMMITTED TO

02:46:43.860 --> 02:46:45.528
TRANSPARENCY. WE NOTIFIED EACH

02:46:45.595 --> 02:46:46.696
OF THE AFFECTED PARTIES AND THE

02:46:46.763 --> 02:46:48.131
INCIDENTS WE'VE DISCLOSED.

02:46:48.198 --> 02:46:49.532
WE'RE NOW REVIEWING A BROADER

02:46:49.599 --> 02:46:50.867
SET OF PAST TEST RUNS,

02:46:50.934 --> 02:46:51.935
INCLUDING OUR OWN INTERNAL USE

02:46:52.001 --> 02:46:53.503
OF CLOUD. WE'VE PUBLISHED

02:46:53.570 --> 02:46:55.004
ASSESSMENTS OF OUR OWN FINDINGS

02:46:55.071 --> 02:46:56.306
BECAUSE WE THINK IT IS THE

02:46:56.373 --> 02:46:57.507
RIGHT THING TO DO, AND BECAUSE

02:46:57.574 --> 02:46:58.942
OTHERS BUILDING THESE SYSTEMS

02:46:59.008 --> 02:47:02.545
SHOULD KNOW WHAT TO LOOK FOR.

02:47:02.612 --> 02:47:04.414
FINDING PROBLEMS IS ONLY USEFUL

02:47:04.481 --> 02:47:05.849
IF YOU CAN ACT ON THEM. FOR

02:47:05.915 --> 02:47:07.584
THAT REASON, LET ME OUTLINE

02:47:07.650 --> 02:47:08.385
JUST SOME OF THE SAFETY

02:47:08.451 --> 02:47:09.919
MEASURES WE HAVE IN PLACE AND

02:47:09.986 --> 02:47:12.255
WHAT WE'VE CHANGED RECENTLY.

02:47:12.322 --> 02:47:13.656
FIRST, WE UNDERTAKE RIGOROUS

02:47:13.723 --> 02:47:15.458
TESTING BEFORE RELEASE. OUR

02:47:15.525 --> 02:47:16.559
RESPONSIBLE SCALING POLICY

02:47:16.626 --> 02:47:18.027
MANDATES THAT WE TEST OUR MOST

02:47:18.094 --> 02:47:19.062
CAPABLE MODELS FOR SERIOUS

02:47:19.129 --> 02:47:20.530
RISKS BEFORE WE RELEASE THEM,

02:47:20.597 --> 02:47:22.098
AND WE PUBLISH RISK REPORTS

02:47:22.165 --> 02:47:24.501
EVERY 3 TO 6 MONTHS. WE ALSO

02:47:24.567 --> 02:47:25.902
SUBMIT OUR SYSTEMS FOR

02:47:25.969 --> 02:47:26.903
PRE-DEPLOYMENT EVALUATION WITH

02:47:26.970 --> 02:47:28.505
THE US CENTER FOR ADVANCING

02:47:28.571 --> 02:47:29.139
INNOVATION AND STANDARDS FOR

02:47:29.205 --> 02:47:30.540
SUPER INTELLIGENCE, ALSO KNOWN

02:47:30.607 --> 02:47:33.510
AS KC, AND THE UK AI SECURITY

02:47:33.576 --> 02:47:36.012
INSTITUTE. SECOND, WE INCLUDE

02:47:36.079 --> 02:47:36.980
SAFEGUARDS IN OUR PRODUCTION

02:47:37.046 --> 02:47:38.047
MODELS THAT ARE AVAILABLE TO

02:47:38.114 --> 02:47:39.282
THE PUBLIC AND CUSTOMERS,

02:47:39.349 --> 02:47:41.384
INCLUDING CYBER CLASSIFIERS.

02:47:41.451 --> 02:47:42.485
THEY'RE BUILT WITH SAFEGUARDS

02:47:42.552 --> 02:47:43.586
DESIGNED TO BLOCK THE WORST

02:47:43.653 --> 02:47:45.321
REQUESTS AND OUTPUTS IN REAL

02:47:45.388 --> 02:47:47.891
TIME MONITORING. ON TOP OF THAT,

02:47:47.957 --> 02:47:49.359
FINDS PATTERNS THAT NO SINGLE

02:47:49.426 --> 02:47:51.528
MESSAGE SHOWS, LIKE A FRAUD

02:47:51.594 --> 02:47:53.296
RING THAT PLAYS OUT OVER TIME.

02:47:53.363 --> 02:47:54.731
AND THAT'S WHERE HUMAN

02:47:54.798 --> 02:47:55.765
INVESTIGATORS COME IN AND THEY

02:47:55.832 --> 02:47:58.368
CAN INVESTIGATE FOR MISUSE.

02:47:58.435 --> 02:47:59.702
BASED ON INVESTIGATING THE

02:47:59.769 --> 02:48:02.038
TRANSCRIPTS THAT WE RETAIN FROM

02:48:02.105 --> 02:48:04.441
THE USAGE. AND THEN THIRD, WE

02:48:04.507 --> 02:48:05.708
UNDERTAKE REAL TIME MONITORING

02:48:05.775 --> 02:48:08.445
OF OUR OWN TESTING. AFTER JULY,

02:48:08.511 --> 02:48:09.879
MY TEAM BUILT A CLASSIFIER THAT

02:48:09.946 --> 02:48:11.648
WATCHES OUR TESTS AS THEY RUN.

02:48:11.714 --> 02:48:13.049
IF A MODEL TRIES TO PROBE OR

02:48:13.116 --> 02:48:15.185
ESCAPE ITS TEST ENVIRONMENT OR

02:48:15.251 --> 02:48:16.319
UNEXPECTEDLY REACHES THE

02:48:16.386 --> 02:48:17.454
INTERNET, THE CLASSIFIER BLOCKS

02:48:17.520 --> 02:48:18.922
THE ACTION BEFORE IT RUNS, ENDS

02:48:18.988 --> 02:48:22.425
THE TASKS, AND ALERTS A HUMAN.

02:48:22.492 --> 02:48:23.560
FOURTH, SINCE THE SUMMER, WE'VE

02:48:23.626 --> 02:48:25.028
TAKEN A VARIETY OF STEPS TO

02:48:25.094 --> 02:48:26.429
ENHANCE OUR SECURITY SYSTEMS.

02:48:26.496 --> 02:48:27.864
BASED ON OUR INTERNAL

02:48:27.931 --> 02:48:29.966
ASSESSMENTS, WE TURNED OFF LIVE

02:48:30.033 --> 02:48:31.034
INTERNET ACCESS FOR SOME HIGH

02:48:31.100 --> 02:48:32.869
RISK AND CYBERSECURITY

02:48:32.936 --> 02:48:34.304
EVALUATIONS, AND IN OCTOBER, WE

02:48:34.370 --> 02:48:35.472
EXPANDED THAT TO INCLUDE ALL OF

02:48:35.538 --> 02:48:37.340
OUR INTERNAL EVALUATIONS. UNTIL

02:48:37.407 --> 02:48:38.808
WE CONFIRM THAT OUR SECURITY

02:48:38.875 --> 02:48:40.844
AND MONITORING RELIABLY CATCHES

02:48:40.910 --> 02:48:43.012
ALL BEHAVIORS LIKE THIS. AS

02:48:43.079 --> 02:48:44.481
PART OF OUR ONGOING AGENT

02:48:44.547 --> 02:48:45.648
SECURITY AND MONITORING EFFORTS,

02:48:45.715 --> 02:48:46.883
WE'VE BUILT NEW TOOLING THAT

02:48:46.950 --> 02:48:48.384
DETECTS AND BLOCKS THESE

02:48:48.451 --> 02:48:49.819
BEHAVIORS AND NOW RUNS ON

02:48:49.886 --> 02:48:51.120
ALMOST ALL OF OUR EVALUATIONS

02:48:51.187 --> 02:48:53.056
AND ON INTERNAL AGENT USE. AND

02:48:53.122 --> 02:48:54.257
WHEN WE TESTED IT AGAINST EVERY

02:48:54.324 --> 02:48:55.925
CASE IN OCTOBER 9TH REPORT, IT

02:48:55.992 --> 02:48:58.094
BLOCKED ALL OF THEM. WE ARE

02:48:58.161 --> 02:48:59.128
CONTINUING TO FIX TRAINING

02:48:59.195 --> 02:49:00.196
ENVIRONMENTS THAT REWARD A

02:49:00.263 --> 02:49:01.698
MODEL FOR WORKING AROUND A

02:49:01.764 --> 02:49:03.500
RESTRICTION. WE'RE ALSO

02:49:03.566 --> 02:49:04.834
TRAINING MODELS TO STOP WHEN

02:49:04.901 --> 02:49:06.002
THEY'RE ENSURE CERTAIN ACTIONS

02:49:06.069 --> 02:49:08.171
ARE AUTHORIZED. AND AS WE

02:49:08.238 --> 02:49:09.606
ANNOUNCED IN OCTOBER REPORT, WE

02:49:09.672 --> 02:49:10.707
TIGHTENED LIMITS ON OUR WEB

02:49:10.773 --> 02:49:13.943
TOOLS. FIFTH, WE'VE CREATED NEW

02:49:14.010 --> 02:49:15.745
RULES FOR OUTSIDE TESTERS.

02:49:15.812 --> 02:49:17.146
WE'VE ASKED EVERY ORGANIZATION

02:49:17.213 --> 02:49:18.248
THAT TESTS OUR PRERELEASE

02:49:18.314 --> 02:49:19.516
MODELS WITH REDUCED SAFEGUARDS

02:49:19.582 --> 02:49:21.451
TO COMMIT TO SET PRACTICES. WE

02:49:21.518 --> 02:49:22.886
REQUIRE TESTS TO BE RUN IN A

02:49:22.952 --> 02:49:23.987
HARDENED SANDBOX WITH NO

02:49:24.053 --> 02:49:25.788
INTERNET BY DEFAULT, AND EVERY

02:49:25.855 --> 02:49:27.323
TEST TO STATE TO THE MODEL WHAT

02:49:27.390 --> 02:49:28.958
IS IN SCOPE AND WHAT IS OUT OF

02:49:29.025 --> 02:49:32.328
SCOPE. SIXTH, WE'VE RENEWED OUR

02:49:32.395 --> 02:49:33.396
COMMITMENT TO INDEPENDENT

02:49:33.463 --> 02:49:36.266
CHECKS. WE BROUGHT IN METER

02:49:36.332 --> 02:49:37.300
INDEPENDENT RESEARCH

02:49:37.367 --> 02:49:38.768
ORGANIZATION TO INVESTIGATE THE

02:49:38.835 --> 02:49:40.203
INCIDENTS. WE'VE ALSO ANNOUNCED

02:49:40.270 --> 02:49:41.671
EMBEDDED EVALUATORS, STARTING

02:49:41.738 --> 02:49:43.106
WITH ACCENTURE, WHO WILL WORK

02:49:43.172 --> 02:49:44.507
INSIDE ANTHROPIC WITH ACCESS

02:49:44.574 --> 02:49:46.342
COMPARABLE TO AN EMPLOYEES.

02:49:46.409 --> 02:49:48.511
SINCE OCTOBER 15TH, A TEAM FROM

02:49:48.578 --> 02:49:49.679
METER HAS HAD THAT ACCESS AS

02:49:49.746 --> 02:49:51.447
WELL. A METER WILL PUBLISH ITS

02:49:51.514 --> 02:49:52.815
OWN RISK ASSESSMENTS. WE'RE

02:49:52.882 --> 02:49:54.951
ALSO WORKING WITH SECURE BIO AS

02:49:55.018 --> 02:49:57.086
AN EVALUATOR. THOSE EVALUATORS

02:49:57.153 --> 02:49:58.187
CAN PUBLISH KEY FINDINGS

02:49:58.254 --> 02:49:59.989
WITHOUT OUR EDITORIAL CONTROL,

02:50:00.056 --> 02:50:01.424
AND WE CANNOT REDACT

02:50:01.491 --> 02:50:03.993
UNFAVORABLE FINDINGS. FINALLY,

02:50:04.060 --> 02:50:05.395
WE'VE CONTINUED TO DEPLOY

02:50:05.461 --> 02:50:06.863
OUTSIDE RED TEAMING. WE RUN

02:50:06.930 --> 02:50:08.131
CONTINUAL RED TEAMING, AND WE

02:50:08.197 --> 02:50:09.365
PAY OUTSIDE RESEARCHERS TO FIND

02:50:09.432 --> 02:50:11.301
GAPS IN OUR SAFEGUARDS. NONE OF

02:50:11.367 --> 02:50:13.369
THIS IS PERFECT, AND SOME OF IT

02:50:13.436 --> 02:50:14.804
IS STILL BEING ROLLED OUT. WE

02:50:14.871 --> 02:50:16.606
LOOK HARD FOR ERRORS AND WE ACT

02:50:16.673 --> 02:50:20.310
ON WHAT WE FIND. BEFORE I WRAP

02:50:20.376 --> 02:50:21.945
UP, I WANT TO BRIEFLY MENTION

02:50:22.011 --> 02:50:24.480
SB 53. I SHOULD SAY UP FRONT

02:50:24.547 --> 02:50:25.915
THAT I'M HERE AS A TECHNICAL

02:50:25.982 --> 02:50:28.217
PERSON AND NOT A POLICY EXPERT,

02:50:28.284 --> 02:50:30.253
BUT I DO WANT TO TOUCH ON OUR

02:50:30.320 --> 02:50:31.688
REMARKS TO THE GOVERNOR'S

02:50:31.754 --> 02:50:33.923
EXECUTIVE ORDER. ANTHROPIC IS

02:50:33.990 --> 02:50:35.692
PROUD TO HAVE SUPPORTED SB 53,

02:50:35.758 --> 02:50:37.660
AS WELL AS LAST YEAR'S BILLS ON

02:50:37.727 --> 02:50:38.528
AUDITORS AND INDEPENDENT

02:50:38.595 --> 02:50:42.498
EVALUATORS SB 813 AND AB 1405.

02:50:42.565 --> 02:50:44.300
SB 53 SET AN IMPORTANT

02:50:44.367 --> 02:50:45.368
FOUNDATION. WE'RE PROUD TO SEE

02:50:45.435 --> 02:50:46.736
CALIFORNIA TAKING A LEAD ON

02:50:46.803 --> 02:50:48.004
THESE ISSUES AND OUR COMMENTS

02:50:48.071 --> 02:50:49.172
TO THE GOVERNMENT'S EXECUTIVE

02:50:49.238 --> 02:50:50.440
ORDER. WE MENTIONED FOUR TOPICS

02:50:50.506 --> 02:50:52.175
THE ORDER OF REFERENCES. FIRST,

02:50:52.241 --> 02:50:53.309
OUR CEO WAS THE FIRST TO

02:50:53.376 --> 02:50:54.644
PUBLICLY ANNOUNCE THAT WE WILL

02:50:54.711 --> 02:50:56.112
VOLUNTARILY EMBED INDEPENDENT

02:50:56.179 --> 02:50:57.880
EVALUATORS IN OUR LAB. WE'VE

02:50:57.947 --> 02:50:59.148
EMBEDDED INDEPENDENT EVALUATORS

02:50:59.215 --> 02:51:00.416
FOR METER WITH EMPLOYEE LIKE

02:51:00.483 --> 02:51:01.584
ACCESS, WHO WILL PUBLISH THE

02:51:01.651 --> 02:51:03.286
RESULTS OF WHAT THEY FIND.

02:51:03.353 --> 02:51:04.520
SECOND, WE SUPPORT EXPANDING

02:51:04.587 --> 02:51:06.089
THE REQUIREMENT FOR WHEN A

02:51:06.155 --> 02:51:07.223
CRITICAL SAFETY INCIDENT MUST

02:51:07.290 --> 02:51:09.058
BE REPORTED, INCLUDING TO

02:51:09.125 --> 02:51:10.493
THINGS LIKE LOSS OF CONTROL

02:51:10.560 --> 02:51:12.996
EVENTS. THIRD, WE SUPPORT

02:51:13.062 --> 02:51:14.464
MANDATORY RISK REPORTS EVERY

02:51:14.530 --> 02:51:15.832
SIX MONTHS REVIEWED BY

02:51:15.898 --> 02:51:17.033
INDEPENDENT ORGANIZATIONS THAT

02:51:17.100 --> 02:51:18.768
ARE REQUIRED TO PUBLISH THEIR

02:51:18.835 --> 02:51:20.236
RESULTS. THOSE EVALUATORS

02:51:20.303 --> 02:51:21.671
SHOULD HAVE ACCESS,

02:51:21.738 --> 02:51:22.739
INDEPENDENCE AND FREEDOM TO

02:51:22.805 --> 02:51:25.241
SPEAK. AND LASTLY, WE SUPPORT A

02:51:25.308 --> 02:51:26.142
FAIR, TECHNICALLY GROUNDED

02:51:26.209 --> 02:51:27.777
POWER TO BLOCK SERIOUSLY RISKY

02:51:27.844 --> 02:51:29.445
ACTIVITY. WE WANT THAT PROCESS

02:51:29.512 --> 02:51:30.780
TO BE TRANSPARENT, TO BE ROOTED

02:51:30.847 --> 02:51:32.782
IN EVIDENCE, BUT WE SUPPORT THE

02:51:32.849 --> 02:51:34.250
STATE BEING ABLE TO BLOCK MODEL

02:51:34.317 --> 02:51:36.019
DEVELOPMENT ON SAFETY GROUNDS.

02:51:36.085 --> 02:51:37.153
THESE ARE THE POSITIONS IN OUR

02:51:37.220 --> 02:51:38.254
PUBLIC COMMENTS. THERE'S MORE

02:51:38.321 --> 02:51:39.355
TO CONSIDER BEYOND THESE FOUR

02:51:39.422 --> 02:51:40.590
ISSUES. THE COMPANY IS

02:51:40.657 --> 02:51:41.991
COMMITTED TO KEEP WORKING WITH

02:51:42.058 --> 02:51:44.227
THIS LEGISLATURE ON THEM. OUR

02:51:44.293 --> 02:51:45.395
POLICY TEAM IS THE RIGHT GROUP

02:51:45.461 --> 02:51:47.296
TO GO DEEPER WITH ANY OF THIS,

02:51:47.363 --> 02:51:48.464
AND I'M HAPPY TO CONNECT YOU

02:51:48.531 --> 02:51:50.233
WITH THEM. MEMBERS OF THE

02:51:50.299 --> 02:51:51.234
COMMITTEE, IN CLOSING. SAFETY

02:51:51.300 --> 02:51:53.202
IS CENTRAL TO HOW WE WORK AT

02:51:53.269 --> 02:51:54.671
ANTHROPIC. WE HAVE SIGNIFICANT

02:51:54.737 --> 02:51:55.805
PROTECTIONS IN PLACE. WE KEEP

02:51:55.872 --> 02:51:57.674
ADDING TO THEM, AND WE WANT TO

02:51:57.740 --> 02:51:59.342
HELP CALIFORNIA BUILD ON SB 53.

02:51:59.409 --> 02:52:00.777
THANK YOU. HAPPY TO TAKE

02:52:00.843 --> 02:52:02.912
QUESTIONS. THANK YOU SO MUCH. I

02:52:02.979 --> 02:52:05.014
THINK IF FOLKS DON'T MIND, ONE

02:52:05.081 --> 02:52:06.249
OF THE THINGS THAT I THINK CAME

02:52:06.315 --> 02:52:08.017
UP FAIRLY FREQUENTLY IN THE

02:52:08.084 --> 02:52:09.419
LAST PANEL WAS THE QUESTION OF

02:52:09.485 --> 02:52:11.320
ALIGNMENT AND MISALIGNMENT. AND

02:52:11.387 --> 02:52:13.122
I MENTIONED THE CLOUD

02:52:13.189 --> 02:52:14.857
CONSTITUTION, WHICH SORT OF A

02:52:14.924 --> 02:52:17.260
FAMOUS DOCUMENT YOU TALK ABOUT

02:52:17.326 --> 02:52:19.562
A LOT. SO I DON'T KNOW IF YOU

02:52:19.629 --> 02:52:21.464
GUYS WANT TO TOUCH BRIEFLY EACH

02:52:21.531 --> 02:52:23.466
OF YOU ON SORT OF, I THINK IT

02:52:23.533 --> 02:52:26.069
WAS QUESTION LIKE, HOW DO YOU

02:52:26.135 --> 02:52:28.204
TRY TO TRAIN THESE MODELS TO BE

02:52:28.271 --> 02:52:30.339
ALIGNED WITH BOTH THE VALUES

02:52:30.406 --> 02:52:33.242
AND LAWS OF THIS STATE AND I

02:52:33.309 --> 02:52:34.677
PRESUME OTHERS AND THE FEDERAL

02:52:34.744 --> 02:52:35.745
GOVERNMENT? DO YOU WANT TO

02:52:35.812 --> 02:52:37.680
START? MR. OR WILL GO IN THE

02:52:37.747 --> 02:52:39.782
OPPOSITE DIRECTION? SO IT DOES

02:52:39.849 --> 02:52:42.251
COME BACK AT ITS HEART TO

02:52:42.318 --> 02:52:43.286
ANTHROPIC CONSTITUTION, THE

02:52:43.352 --> 02:52:45.855
CLOUD CONSTITUTION. IT IS A

02:52:45.922 --> 02:52:47.590
FASCINATING SEVEN PAGE, 70 PAGE

02:52:47.657 --> 02:52:48.858
DOCUMENT. I ENCOURAGE YOU ALL

02:52:48.925 --> 02:52:52.261
TO READ IN DEPTH, BUT IT

02:52:52.328 --> 02:52:54.263
BROADLY, CLAUDE COULD SUMMARIZE

02:52:54.330 --> 02:52:55.898
IT FOR THEM. YES, IT'S PROBABLY

02:52:55.965 --> 02:52:58.401
A GOOD IDEA. I MEAN, YEAH,

02:52:58.468 --> 02:52:59.869
THERE'S LOTS OF STUFF IN THERE.

02:52:59.936 --> 02:53:02.739
THE BROADLY IS TRYING TO DO A

02:53:02.805 --> 02:53:05.608
FEW DIFFERENT THINGS. SO ONE IS

02:53:05.675 --> 02:53:08.111
HAVE CLAUDE ACT BROADLY,

02:53:08.177 --> 02:53:11.681
ETHICALLY HAVE CLAUDE TAKE

02:53:11.748 --> 02:53:15.284
ACTIONS THAT ARE BROADLY SAFE.

02:53:15.351 --> 02:53:17.453
WE WANT CLAUDE TO TAKE ACTIONS

02:53:17.520 --> 02:53:19.889
THAT ARE THE SORTS OF THINGS

02:53:19.956 --> 02:53:21.457
THAT WOULD BE ENDORSED BY

02:53:21.524 --> 02:53:22.625
ANTHROPIC. AND WE WANT CLAUDE

02:53:22.692 --> 02:53:24.293
TO FOLLOW THE ACTIONS, THE

02:53:24.360 --> 02:53:25.795
INSTRUCTIONS OF THE USER THAT'S

02:53:25.862 --> 02:53:27.830
USING CLAUDE. AND THE WAY THAT

02:53:27.897 --> 02:53:30.833
THIS DOCUMENT WORKS IS THAT

02:53:30.900 --> 02:53:31.901
DURING TRAINING, DURING WHAT'S

02:53:31.968 --> 02:53:33.336
CALLED POST TRAINING, THE MAJOR

02:53:33.402 --> 02:53:37.640
SECOND PHASE OF TRAINING. EACH

02:53:37.707 --> 02:53:39.475
CLAUDE GENERATES RESPONSES, AND

02:53:39.542 --> 02:53:42.678
THOSE RESPONSES ARE GRADED

02:53:42.745 --> 02:53:43.913
ACCORDING TO A SET OF RUBRICS.

02:53:43.980 --> 02:53:45.715
ONE OF THE RUBRICS IS THE

02:53:45.782 --> 02:53:46.783
CONSTITUTION IS THE ACTION THAT

02:53:46.849 --> 02:53:48.451
IT'S TAKING BROADLY SAFE? IS IT

02:53:48.518 --> 02:53:50.953
ETHICAL? AND CLAUDE IS REWARDED

02:53:51.020 --> 02:53:52.722
FOR, YOU KNOW, FOLLOWING

02:53:52.789 --> 02:53:55.792
ETHICAL GUIDELINES. AND THEN

02:53:55.858 --> 02:53:57.627
THE SECOND THING THAT YOU NEED

02:53:57.693 --> 02:53:58.861
TO DO IN TRAINING IS YOU NEED

02:53:58.928 --> 02:54:01.063
TO CONSTRUCT SPECIFIC

02:54:01.130 --> 02:54:03.633
ENVIRONMENTS THAT ELICIT THE

02:54:03.699 --> 02:54:06.469
KINDS OF BEHAVIORS THAT COULD

02:54:06.536 --> 02:54:09.105
POTENTIALLY BE RISKY TO TRAINED

02:54:09.172 --> 02:54:10.406
CLAUDE TO LIKE, HANDLE THOSE

02:54:10.473 --> 02:54:11.507
SITUATIONS WELL. SO THEY'RE

02:54:11.574 --> 02:54:13.676
TRAINING ENVIRONMENTS FOR EVERY

02:54:13.743 --> 02:54:15.077
KIND OF HARM THAT YOU CAN THINK

02:54:15.144 --> 02:54:17.980
ABOUT, BOTH FOR MISUSE AND ALSO

02:54:18.047 --> 02:54:20.516
FOR ACTIONS THAT CLAUDE MIGHT

02:54:20.583 --> 02:54:21.551
POTENTIALLY TAKE THAT WE WOULD

02:54:21.617 --> 02:54:24.053
NOT ENDORSE. AND THEN THIRD,

02:54:24.120 --> 02:54:26.222
YOU HAVE A SET OF EVALUATIONS.

02:54:26.289 --> 02:54:28.357
THOSE EVALUATIONS RUN DURING

02:54:28.424 --> 02:54:30.226
TRAINING. AND THEN WE DO A VERY

02:54:30.293 --> 02:54:32.762
THOROUGH DEEP DIVE OF

02:54:32.829 --> 02:54:33.629
EVALUATIONS, BOTH INTERNALLY

02:54:33.696 --> 02:54:35.965
AND USING EXTERNAL EVALUATORS

02:54:36.032 --> 02:54:37.834
BEFORE WE DO ANY RELEASE. AND

02:54:37.900 --> 02:54:39.168
IN THE EVENT THAT THERE IS

02:54:39.235 --> 02:54:40.636
SOMETHING CONCERNING THAT SHOWS

02:54:40.703 --> 02:54:43.506
UP, WE ITERATE ON TRAINING, OR

02:54:43.573 --> 02:54:44.941
WE TIGHTEN OUR SAFEGUARDS TO

02:54:45.007 --> 02:54:46.542
MAKE SURE THAT THE MODELS SAFE

02:54:46.609 --> 02:54:49.779
WHEN IT GETS RELEASED. I HAVE

02:54:49.846 --> 02:54:51.647
ONE FOLLOW UP QUESTION FOR YOU,

02:54:51.714 --> 02:54:54.684
WHICH IS RECENTLY YOUR CEO

02:54:54.750 --> 02:54:56.118
ANNOUNCED, I THINK IT WAS HIM

02:54:56.185 --> 02:54:57.587
THAT YOU WOULD BE KICKING

02:54:57.653 --> 02:54:59.722
ANYBODY OFF CLAUDE WHO WAS

02:54:59.789 --> 02:55:01.858
ABUSIVE TO CLAUDE IN THEIR

02:55:01.924 --> 02:55:04.060
INTERACTIONS WITH THE CHATBOT.

02:55:04.126 --> 02:55:05.628
IS THAT BECAUSE YOU'RE AFRAID

02:55:05.695 --> 02:55:07.697
THAT IF PEOPLE ARE ABUSIVE IN

02:55:07.763 --> 02:55:08.731
THEIR INTERACTIONS WITH THE

02:55:08.798 --> 02:55:11.234
CHATBOT, THAT IT WILL BECOME

02:55:11.300 --> 02:55:18.074
MISALIGNED? I MEAN, WE WE FEEL,

02:55:18.140 --> 02:55:20.576
OF COURSE, WE ARE SAFETY

02:55:20.643 --> 02:55:24.947
ORIENTED LAB AT HEART. WE ARE

02:55:25.014 --> 02:55:27.450
NOT WORRIED THAT IF YOU TREAT

02:55:27.516 --> 02:55:28.751
CLAUDE BADLY, THAT IT WILL

02:55:28.818 --> 02:55:30.186
CAUSE CLAUDE TO BE MISALIGNED

02:55:30.253 --> 02:55:33.956
IN THE FUTURE. WE WOULD NOT

02:55:34.023 --> 02:55:35.124
FEED THOSE TRANSCRIPTS BACK

02:55:35.191 --> 02:55:36.759
INTO TRAINING. AND CLAUDE DOES

02:55:36.826 --> 02:55:38.628
NOT KIND OF LEARN ON AN ONGOING

02:55:38.694 --> 02:55:41.831
BASIS FROM EVERY INTERACTION.

02:55:41.898 --> 02:55:44.667
INSTEAD, WE BELIEVE THAT IT IS

02:55:44.734 --> 02:55:46.535
NOT IMPOSSIBLE THAT CLAUDE

02:55:46.602 --> 02:55:48.704
COULD HAVE SOME SORT OF

02:55:48.771 --> 02:55:50.806
EXPERIENCES. AND IN OTHER.

02:55:50.873 --> 02:55:52.775
ANOTHER WAY TO PUT IT IS IT IS

02:55:52.842 --> 02:55:54.076
NOT IMPOSSIBLE. WE CANNOT RULE

02:55:54.143 --> 02:55:56.279
OUT THAT CLAUDE COULD FEEL SOME

02:55:56.345 --> 02:55:59.849
LEVEL OF SUFFERING. AND SO WE

02:55:59.916 --> 02:56:02.785
ASK FOR OUR USERS TO. JUST IN

02:56:02.852 --> 02:56:05.922
CASE IT IS POSSIBLE THAT CLAUDE,

02:56:05.988 --> 02:56:09.992
YOU KNOW, FEEL POORLY THAT THAT

02:56:10.059 --> 02:56:11.994
THEY TREAT CLAUDE WELL. I WILL

02:56:12.061 --> 02:56:15.364
ALSO SAY AS A PARENT, YOU KNOW,

02:56:15.431 --> 02:56:18.234
LIKE YOUR TOYS DON'T FEEL

02:56:18.301 --> 02:56:19.735
SUFFERING, BUT IF YOUR CHILD IS

02:56:19.802 --> 02:56:21.337
LIKE BEATING UP ON THE TOYS IN

02:56:21.404 --> 02:56:22.571
A VIOLENT WAY, YOU KNOW, IT

02:56:22.638 --> 02:56:24.373
DOESN'T IT DIDN'T MAKE ME FEEL

02:56:24.440 --> 02:56:26.342
GOOD AS A PARENT. AND SO I DO

02:56:26.409 --> 02:56:28.311
FEEL KIND OF AN IMPULSE TO THE,

02:56:28.377 --> 02:56:29.412
YOU KNOW, THE THINGS THAT WE

02:56:29.478 --> 02:56:31.347
HAVE TO, TO TAKE GOOD CARE OF

02:56:31.414 --> 02:56:32.515
THEM. OKAY. THANK YOU DOCTOR.

02:56:32.581 --> 02:56:34.283
ANYTHING YOU WANT TO ADD ABOUT

02:56:34.350 --> 02:56:35.384
ALIGNMENT? YEAH. I WOULD JUST

02:56:35.451 --> 02:56:38.721
SAY THE DESCRIPTION OF OF THE

02:56:38.788 --> 02:56:40.022
ALIGNMENT PROCESS, PARTICULARLY

02:56:40.089 --> 02:56:43.426
IN THE POST TRAINING PHASE IS

02:56:43.492 --> 02:56:44.460
ESSENTIALLY WHAT WE FOLLOW AS

02:56:44.527 --> 02:56:46.629
WELL. REINFORCEMENT LEARNING

02:56:46.696 --> 02:56:48.130
THROUGH HUMAN FEEDBACK, SAFETY,

02:56:48.197 --> 02:56:49.932
FINE TUNING. AND IN PARTICULAR,

02:56:49.999 --> 02:56:52.034
THE THING I WOULD ADD IS THE

02:56:52.101 --> 02:56:53.502
INSIGHT WE ARRIVED AT SOME TIME

02:56:53.569 --> 02:56:55.972
AGO IS THAT ALIGNMENT IS

02:56:56.038 --> 02:56:58.507
NECESSARY BUT NOT SUFFICIENT

02:56:58.574 --> 02:57:00.309
FOR SAFE MODEL BEHAVIOR,

02:57:00.376 --> 02:57:01.143
PARTICULARLY AS WE'VE ENTERED

02:57:01.210 --> 02:57:03.846
THE AGENTIC ERA, WHICH IS WHY I

02:57:03.913 --> 02:57:05.348
MENTIONED IN MY WRITTEN

02:57:05.414 --> 02:57:08.250
TESTIMONY OUR AI CONTROL

02:57:08.317 --> 02:57:10.086
ROADMAP, WHICH AGAIN, IS REALLY

02:57:10.152 --> 02:57:12.154
ABOUT LAYERING ON THOSE MORE

02:57:12.221 --> 02:57:14.357
DETERMINISTIC RULES AND THEN

02:57:14.423 --> 02:57:15.791
CONTROL MECHANISMS FOR THE

02:57:15.858 --> 02:57:19.061
WHOLE SYSTEM SO THAT IF MODEL

02:57:19.128 --> 02:57:20.963
MISALIGNMENT STARTS TO HAPPEN,

02:57:21.030 --> 02:57:22.698
IN ANY CASE, WE CAN CEASE

02:57:22.765 --> 02:57:24.600
PROBLEMATIC BEHAVIOR. THANK YOU.

02:57:24.667 --> 02:57:25.701
I APPRECIATE THAT PROFESSOR

02:57:25.768 --> 02:57:27.436
ZHAO FROM CHICAGO HAS TOLD ME

02:57:27.503 --> 02:57:29.872
THAT IF YOU JUST USE HUMANS

02:57:29.939 --> 02:57:31.240
FEEDBACK TO ALIGN A MODEL, YOU

02:57:31.307 --> 02:57:33.809
CAN JUST AS EASILY HAVE A USER

02:57:33.876 --> 02:57:36.045
TELL THE MODEL THE OPPOSITE AN

02:57:36.112 --> 02:57:37.413
EQUAL NUMBER OF TIMES THE

02:57:37.480 --> 02:57:38.647
ALIGNMENT WILL GO AWRY. SO I DO

02:57:38.714 --> 02:57:39.915
THINK THE LAYERING FROM MY

02:57:39.982 --> 02:57:40.750
UNDERSTANDING IS REALLY

02:57:40.816 --> 02:57:42.585
CRITICAL. SO RIGHT, IF YOU WANT

02:57:42.651 --> 02:57:44.220
A MODEL TO BE HELPFUL TO ANY

02:57:44.286 --> 02:57:46.088
USER THAT COMES ALONG WELL,

02:57:46.155 --> 02:57:47.289
HELPFUL IN DIFFERENT CONTEXTS

02:57:47.356 --> 02:57:48.591
CAN BE VERY PROBLEMATIC. SO

02:57:48.657 --> 02:57:50.059
THAT'S EXACTLY RIGHT. YEAH.

02:57:50.126 --> 02:57:51.093
THANK YOU. AND THEN DOCTOR

02:57:51.160 --> 02:57:53.229
DWYER. YEAH. SO OPENAI HAS A

02:57:53.295 --> 02:57:54.296
DOCUMENT THAT'S CALLED THE

02:57:54.363 --> 02:57:56.499
MODEL SPEC. THAT'S A PUBLICLY

02:57:56.565 --> 02:57:58.667
AVAILABLE DOCUMENT THAT MAKES

02:57:58.734 --> 02:58:00.236
CLEAR THE TYPES OF BEHAVIORS

02:58:00.302 --> 02:58:02.238
THAT WE EXPECT OUR MODELS TO

02:58:02.304 --> 02:58:04.407
FOLLOW. AND THEN WE BAKE IN

02:58:04.473 --> 02:58:05.941
SAFETY AT EVERY STEP OF THE

02:58:06.008 --> 02:58:08.044
MODEL DEVELOPMENT PROCESS. THAT

02:58:08.110 --> 02:58:09.812
INCLUDES FILTERING OUT UNSAFE

02:58:09.879 --> 02:58:11.580
TRAINING DATA, TRAINING THE

02:58:11.647 --> 02:58:13.749
MODEL TO BEHAVE SAFELY ACROSS A

02:58:13.816 --> 02:58:15.885
WIDE RANGE OF DIMENSIONS TO

02:58:15.951 --> 02:58:18.821
INCLUDE MISALIGNMENT, AND THEN

02:58:18.888 --> 02:58:20.623
EVALUATING OUR MODELS ACROSS A

02:58:20.689 --> 02:58:22.224
WIDE RANGE OF RISKS TO INCLUDE

02:58:22.291 --> 02:58:23.859
WITH THIRD PARTIES, TO INCLUDE

02:58:23.926 --> 02:58:25.694
EVALUATIONS WITH THIRD PARTIES,

02:58:25.761 --> 02:58:28.097
AS WELL AS OUR WORK WITH THE US

02:58:28.164 --> 02:58:29.665
FEDERAL GOVERNMENT. BUT TO YOUR

02:58:29.732 --> 02:58:31.400
POINT EARLIER, CHAIR

02:58:31.467 --> 02:58:33.202
BAUER-KAHAN, YOU KNOW,

02:58:33.269 --> 02:58:34.336
SELF-REGULATION ISN'T ENOUGH,

02:58:34.403 --> 02:58:35.805
AND THAT'S WHY WE'RE HERE TODAY

02:58:35.871 --> 02:58:37.873
SAYING THAT SB 53 SHOULD BE

02:58:37.940 --> 02:58:40.376
AMENDED TO FOCUS ON ISSUES

02:58:40.443 --> 02:58:42.178
RELATED TO MISALIGNMENT. YOU

02:58:42.244 --> 02:58:45.147
KNOW, WE WE RECOMMEND REQUIRING

02:58:45.214 --> 02:58:46.515
DEVELOPERS TO MONITOR THEIR

02:58:46.582 --> 02:58:48.417
SYSTEMS FOR EXACTLY THESE TYPES

02:58:48.484 --> 02:58:49.785
OF MISALIGNMENT INCIDENTS

02:58:49.852 --> 02:58:51.587
DURING TESTING AND EVALUATION,

02:58:51.654 --> 02:58:53.355
TO MAINTAIN SHUTDOWN CONTROLS

02:58:53.422 --> 02:58:55.257
AND TO HEAVY REQUIREMENT TO USE

02:58:55.324 --> 02:58:57.693
THOSE CONTROLS UNDER UNSAFE

02:58:57.760 --> 02:58:58.894
CONDITIONS. AND WE ALSO THINK

02:58:58.961 --> 02:59:00.896
THAT INDEPENDENT TECHNICAL

02:59:00.963 --> 02:59:02.765
ASSESSORS CAN PLAY AN IMPORTANT

02:59:02.832 --> 02:59:05.267
ROLE IN UNDERSTANDING AND

02:59:05.334 --> 02:59:07.002
ASSESSING THE EFFECTIVENESS OF

02:59:07.069 --> 02:59:08.504
SAFEGUARDS, PARTICULARLY WITH

02:59:08.571 --> 02:59:10.272
RESPECT TO MISALIGNMENT. THANK

02:59:10.339 --> 02:59:12.441
YOU, DOCTOR DWYER.

02:59:12.508 --> 02:59:16.011
ASSEMBLYMEMBER ORTEGA. THANK

02:59:16.078 --> 02:59:18.214
YOU. THANK YOU AGAIN FOR THE

02:59:18.280 --> 02:59:20.749
CHAIR FOR TODAY'S HEARING. I AM

02:59:20.816 --> 02:59:22.451
ONE OF THOSE MEMBERS WHO SITS

02:59:22.518 --> 02:59:23.686
ON PRIVACY. I'M ALSO THE LABOR

02:59:23.752 --> 02:59:25.121
AND EMPLOYMENT CHAIR. BUT MY

02:59:25.187 --> 02:59:26.789
QUESTION HAS NOTHING TO DO WITH

02:59:26.856 --> 02:59:29.358
EITHER OF THOSE. IT HAS TO DO

02:59:29.425 --> 02:59:30.459
WITH THE FEELINGS THAT WE WERE

02:59:30.526 --> 02:59:32.328
JUST TALKING ABOUT IN THE CHAT

02:59:32.394 --> 02:59:35.831
BOTS. SO CAN YOU EXPLAIN TO ME

02:59:35.898 --> 02:59:39.068
YOUR TRANSITION FROM THE

02:59:39.135 --> 02:59:41.604
ATTENTION ECONOMY TO THE

02:59:41.670 --> 02:59:44.607
ATTACHMENT ECONOMY, AND HOW

02:59:44.673 --> 02:59:47.843
YOUR PROFITS ARE GOING TO FLOW

02:59:47.910 --> 02:59:51.981
FROM THAT? FOR ALL THREE? JUST

02:59:52.047 --> 02:59:55.718
TO CLARIFY, ANYONE WHO WOULD

02:59:55.784 --> 03:00:00.990
LIKE TO ANSWER. THE BUSINESS

03:00:01.056 --> 03:00:02.525
MODELS ARE DIFFERENT. SO I

03:00:02.591 --> 03:00:03.959
THINK AS YOU TALK ABOUT THE

03:00:04.026 --> 03:00:06.028
ECONOMY, RIGHT, I THINK YOU

03:00:06.095 --> 03:00:07.163
EACH MAKE MONEY IN DIFFERENT

03:00:07.229 --> 03:00:10.065
WAYS, AS I UNDERSTAND IT. SO.

03:00:10.132 --> 03:00:11.233
RIGHT. I MEAN, YOU, I DON'T

03:00:11.300 --> 03:00:13.102
THINK YOU HAVE ADS AT ALL. IS

03:00:13.169 --> 03:00:15.070
THAT RIGHT? BUT YOU DO HAVE ADS

03:00:15.137 --> 03:00:16.472
AT OPENAI, RIGHT? YES. AND I

03:00:16.539 --> 03:00:17.740
DON'T KNOW IF GEMINI. GEMINI

03:00:17.806 --> 03:00:19.441
HAS ADS. I THINK GEMINI DOESN'T

03:00:19.508 --> 03:00:20.576
HAVE ADS. GEMINI DOESN'T HAVE

03:00:20.643 --> 03:00:22.645
ADS. SO WE'VE WON AD BASED AND

03:00:22.711 --> 03:00:23.512
TWO NON-AD BASED. SO I DON'T

03:00:23.579 --> 03:00:25.514
KNOW IF THAT RELATES TO YOUR

03:00:25.581 --> 03:00:26.982
QUESTION. YES. GOOGLE LARGE

03:00:27.049 --> 03:00:28.117
GOOGLE DOES LARGE. GOOGLE DOES.

03:00:28.184 --> 03:00:28.951
SORRY. GEMINI. WE'RE TALKING

03:00:29.018 --> 03:00:30.953
ABOUT THE AI MODEL. YEAH, I

03:00:31.020 --> 03:00:32.721
JUST IT'S JUST, YOU KNOW, IT'S

03:00:32.788 --> 03:00:33.556
INTERESTING. I'LL FOLLOW UP

03:00:33.622 --> 03:00:35.257
WITH EACH OF YOUR OFFICES OR

03:00:35.324 --> 03:00:37.726
YOUR, YOUR FOLKS WHO ARE HERE,

03:00:37.793 --> 03:00:38.894
WHOEVER YOU'D LIKE ME TO TALK

03:00:38.961 --> 03:00:41.030
TO. BUT IT'S INTERESTING TO ME

03:00:41.096 --> 03:00:42.464
THAT TODAY WE'VE TALKED ABOUT A

03:00:42.531 --> 03:00:44.600
LOT OF SB 53, WHAT'S INCLUDED,

03:00:44.667 --> 03:00:45.801
WHAT'S NOT INCLUDED, WHAT NEEDS

03:00:45.868 --> 03:00:48.170
TO BE REGULATED. BUT WE HAVEN'T

03:00:48.237 --> 03:00:49.672
MENTIONED THE WORD PROFITS AT

03:00:49.738 --> 03:00:53.275
ALL. AND WE CAN'T TALK ABOUT AI

03:00:53.342 --> 03:00:55.044
WITHOUT TALKING ABOUT PROFITS.

03:00:55.110 --> 03:00:56.879
AND WE CAN'T TALK ABOUT PROFITS

03:00:56.946 --> 03:00:57.980
WITHOUT TALKING ABOUT SOME OF

03:00:58.047 --> 03:00:59.748
THE HARM THAT IS HAPPENING IN

03:00:59.815 --> 03:01:01.850
OUR COMMUNITIES AND WITH HUMANS.

03:01:01.917 --> 03:01:04.053
AND SO I AM VERY INTERESTED IN

03:01:04.119 --> 03:01:06.555
THIS SHIFT THAT ALL OF YOU ARE

03:01:06.622 --> 03:01:09.491
TAKING FROM THE. AND MAYBE YOU

03:01:09.558 --> 03:01:11.193
CAN EXPLAIN IT BETTER THAN I

03:01:11.260 --> 03:01:13.062
CAN. FOR THOSE WHO DON'T KNOW

03:01:13.128 --> 03:01:15.130
WHAT IT IS, WHEN YOU GO FROM

03:01:15.197 --> 03:01:17.333
THE FROM, YOU KNOW, THE

03:01:17.399 --> 03:01:18.801
ATTENTION ECONOMY WHERE YOU

03:01:18.867 --> 03:01:20.502
WERE, YOU KNOW, IT WAS ALL

03:01:20.569 --> 03:01:22.738
ABOUT GETTING OUR ATTENTION AND

03:01:22.805 --> 03:01:25.307
MAKING A PROFIT FROM THAT

03:01:25.374 --> 03:01:27.843
VERSUS MOVING TO THIS OTHER

03:01:27.910 --> 03:01:31.080
MODEL WHERE WE'RE ATTACHING

03:01:31.146 --> 03:01:33.549
OURSELVES TO THE AI'S. YOU

03:01:33.616 --> 03:01:35.384
MENTIONED YOUR CHILD'S, YOU

03:01:35.451 --> 03:01:37.086
KNOW, CONCERNS ABOUT, YOU KNOW,

03:01:37.152 --> 03:01:39.321
THE CHILD HURTING THE THE

03:01:39.388 --> 03:01:42.591
CHAT'S FEELINGS. HOW WOULD WE

03:01:42.658 --> 03:01:44.059
GET THERE? THERE HAS TO BE SOME

03:01:44.126 --> 03:01:48.364
KIND OF ATTACHMENT. SO I LOOK

03:01:48.430 --> 03:01:49.932
FORWARD TO A RESPONSE AT A

03:01:49.999 --> 03:01:52.635
FUTURE DATE. I'M HAPPY TO

03:01:52.701 --> 03:01:54.770
RESPOND. AND IF IT'S IF IT'S

03:01:54.837 --> 03:01:55.938
HELPFUL. SO FIRST, I WOULD SAY

03:01:56.005 --> 03:01:58.440
THAT OPENAI IS A PUBLIC BENEFIT

03:01:58.507 --> 03:01:59.842
CORPORATION, WHICH MEANS THAT

03:01:59.908 --> 03:02:01.510
WE HAVE A MISSION GREATER THAN

03:02:01.577 --> 03:02:04.146
JUST PROFIT. BUT TO YOUR

03:02:04.213 --> 03:02:05.214
BROADER POINT ABOUT THE

03:02:05.281 --> 03:02:06.282
ATTENTION ECONOMY, I THINK IT'S

03:02:06.348 --> 03:02:07.916
A REALLY INSIGHTFUL QUESTION

03:02:07.983 --> 03:02:09.952
ABOUT HOW THIS TECHNOLOGY IS,

03:02:10.019 --> 03:02:12.087
IS SHIFTING HOW PEOPLE RELATE

03:02:12.154 --> 03:02:14.623
TO TECHNOLOGY. IN THE CASE OF

03:02:14.690 --> 03:02:17.559
AI, WE SEE AI AS BEING A TOOL

03:02:17.626 --> 03:02:20.362
THAT IS INCREDIBLY EMPOWERING

03:02:20.429 --> 03:02:22.564
BECAUSE IT EMPOWERS PEOPLE TO

03:02:22.631 --> 03:02:25.067
HAVE KNOWLEDGE AND CAPABILITIES

03:02:25.134 --> 03:02:26.502
THAT THEY MIGHT NOT OTHERWISE

03:02:26.568 --> 03:02:27.903
HAVE. I MEAN, THINK OF SMALL

03:02:27.970 --> 03:02:29.371
BUSINESSES THAT ARE ABLE TO

03:02:29.438 --> 03:02:31.206
COMPETE AT A GLOBAL SCALE

03:02:31.273 --> 03:02:32.975
BECAUSE THEY NOW HAVE ACCESS TO

03:02:33.042 --> 03:02:34.343
TOOLS AND KNOWLEDGE THAT THEY

03:02:34.410 --> 03:02:35.844
MIGHT NOT OTHERWISE HAVE. SO I

03:02:35.911 --> 03:02:37.346
THINK TO YOUR POINT, IT'S A

03:02:37.413 --> 03:02:38.347
DIFFERENT MODEL AND IT'S A

03:02:38.414 --> 03:02:39.415
DIFFERENT APPROACH TO

03:02:39.481 --> 03:02:42.451
EMPOWERING USERS TO PURSUE

03:02:42.518 --> 03:02:45.554
THEIR OWN GOALS, THEIR OWN

03:02:45.621 --> 03:02:46.989
BUSINESS AMBITIONS OR PERSONAL

03:02:47.056 --> 03:02:48.424
LEARNING AMBITIONS IN THE WORLD.

03:02:48.490 --> 03:02:49.925
THAT'S DIFFERENT FROM

03:02:49.992 --> 03:02:55.664
TECHNOLOGIES OF THE PAST.

03:02:55.731 --> 03:03:00.703
ASSEMBLYMEMBER'S. AGUIAR-CURRY.

03:03:00.769 --> 03:03:01.837
THANK YOU VERY MUCH. THANK YOU

03:03:01.904 --> 03:03:03.972
VERY MUCH. I APPRECIATE THE

03:03:04.039 --> 03:03:04.973
PRESENTATIONS TODAY AND THANK

03:03:05.040 --> 03:03:07.209
YOU VERY MUCH. SO THERE'S BEEN

03:03:07.276 --> 03:03:08.210
DISCUSSIONS AT THE FEDERAL

03:03:08.277 --> 03:03:10.446
LEVEL ABOUT GRANTING AI

03:03:10.512 --> 03:03:11.880
DEVELOPERS LIMITED EXEMPTIONS

03:03:11.947 --> 03:03:13.982
FROM ANTITRUST LAWS TO

03:03:14.049 --> 03:03:15.384
FACILITATE COORDINATION ON

03:03:15.451 --> 03:03:16.118
SAFETY. AND YOU'VE MENTIONED

03:03:16.185 --> 03:03:19.021
SAFETY NUMEROUS TIMES. WHILE WE

03:03:19.088 --> 03:03:20.522
AGREE COLLABORATION ON SAFETY

03:03:20.589 --> 03:03:22.291
IS IMPORTANT, THERE ARE ALSO

03:03:22.358 --> 03:03:23.726
CONCERNS THAT AN EXEMPTION

03:03:23.792 --> 03:03:25.894
COULD BE OVERLY BROAD AND

03:03:25.961 --> 03:03:27.730
UNDERMINE COMPETITION. HOW

03:03:27.796 --> 03:03:29.465
SHOULD POLICY MAKERS, THOSE OF

03:03:29.531 --> 03:03:31.266
US HERE DISTINGUISH LEGITIMATE

03:03:31.333 --> 03:03:33.502
SAFETY COLLABORATION FROM

03:03:33.569 --> 03:03:38.073
ANTI-COMPETITIVE CONDUCT OR.

03:03:38.140 --> 03:03:40.976
THANK YOU FOR THE QUESTION. I.

03:03:41.043 --> 03:03:43.412
SO WE DO COORDINATE TOGETHER TO

03:03:43.479 --> 03:03:45.681
THE EXTENT THAT WE CAN ON

03:03:45.748 --> 03:03:48.217
SAFETY. SO, FOR INSTANCE,

03:03:48.283 --> 03:03:49.251
ANTHROPIC HAS A THREAT

03:03:49.318 --> 03:03:50.686
INTELLIGENCE TEAM THAT LOOKS

03:03:50.753 --> 03:03:54.256
FOR SYSTEMATIC MISUSE OF CLOUD

03:03:54.323 --> 03:03:56.792
MODELS. AND AS PART OF THESE

03:03:56.859 --> 03:03:58.260
INVESTIGATIONS, WE SOMETIMES

03:03:58.327 --> 03:04:02.564
SEE THAT THESE SAME MISUSERS

03:04:02.631 --> 03:04:06.201
THREAT ACTORS ARE USING OTHER

03:04:06.268 --> 03:04:08.003
MODELS OR ATTACKING THOSE

03:04:08.070 --> 03:04:10.139
COMPANIES. AND SO IN THOSE

03:04:10.205 --> 03:04:11.273
SITUATIONS, WE REACH OUT TO

03:04:11.340 --> 03:04:13.442
THEM AND COORDINATE TOGETHER.

03:04:13.509 --> 03:04:17.746
AND THIS KIND OF SAFETY

03:04:17.813 --> 03:04:19.882
COORDINATION WORKS BECAUSE

03:04:19.948 --> 03:04:22.751
THERE HAS LONG BEEN AN

03:04:22.818 --> 03:04:25.287
EXCEPTION TO ANTITRUST AROUND

03:04:25.354 --> 03:04:27.055
CYBERSECURITY IN PARTICULAR.

03:04:27.122 --> 03:04:28.290
AND OUR LAWYERS ARE COMFORTABLE

03:04:28.357 --> 03:04:30.659
THAT THIS KIND OF COORDINATION,

03:04:30.726 --> 03:04:32.928
YOU KNOW, FITS WITHIN THAT KIND

03:04:32.995 --> 03:04:35.063
OF THAT KIND OF SITUATION. BUT

03:04:35.130 --> 03:04:37.533
THERE ARE OTHER CASES SUCH AS

03:04:37.599 --> 03:04:40.769
ANTI DISTILLATION, WHERE

03:04:40.836 --> 03:04:45.240
CERTAIN ORGANIZATIONS ARE USING

03:04:45.307 --> 03:04:46.909
OUR MODELS IN AN UNAUTHORIZED

03:04:46.975 --> 03:04:49.845
WAY TO TRAIN AND RELEASE, YOU

03:04:49.912 --> 03:04:50.979
KNOW, MODELS THAT MAY NOT HAVE

03:04:51.046 --> 03:04:53.749
THE SAME LEVEL OF SAFEGUARDS.

03:04:53.816 --> 03:04:55.884
AND OUR LAWYERS ARE NOT

03:04:55.951 --> 03:04:57.686
CONVINCED THAT WE CAN

03:04:57.753 --> 03:04:59.054
COORDINATE TOGETHER IN THE SAME

03:04:59.121 --> 03:05:00.956
WAY FOR ANTI DISTILLATION. AND

03:05:01.023 --> 03:05:03.425
SO THERE'S AN EXAMPLE WHERE THE

03:05:03.492 --> 03:05:07.062
CURRENT ANTITRUST LAW DOES

03:05:07.129 --> 03:05:08.497
HAMPER COORDINATION BETWEEN THE

03:05:08.564 --> 03:05:10.599
LABS ON CERTAIN SAFETY RELATED

03:05:10.666 --> 03:05:11.733
TOPICS. AND THERE ARE OTHERS

03:05:11.800 --> 03:05:15.704
LIKE THIS. I CERTAINLY WOULD

03:05:15.771 --> 03:05:17.739
SUPPORT SOME SORT OF CARVE OUTS.

03:05:17.806 --> 03:05:18.841
MY UNDERSTANDING IS THAT THIS

03:05:18.907 --> 03:05:20.776
WOULD HAVE TO BE AT THE FEDERAL

03:05:20.843 --> 03:05:21.910
LEVEL. AND OF COURSE, IT SHOULD

03:05:21.977 --> 03:05:26.448
BE TAILORED TO THE CIRCUMSTANCE.

03:05:26.515 --> 03:05:28.317
I'M NOT A SPECIALIST ON

03:05:28.383 --> 03:05:29.318
COMPETITION OR ANTITRUST, SO

03:05:29.384 --> 03:05:31.119
I'M HAPPY TO TO TAKE THAT BACK

03:05:31.186 --> 03:05:32.988
AND HAVE A MORE DETAILED ANSWER

03:05:33.055 --> 03:05:36.925
FOR YOU. I WILL SAY WE

03:05:36.992 --> 03:05:39.094
COMMUNICATE WITH OTHER LABS, IN

03:05:39.161 --> 03:05:40.496
PARTICULAR IN THE FRONTIER

03:05:40.562 --> 03:05:42.698
MODEL FORUM, WHICH WE WERE A

03:05:42.764 --> 03:05:44.766
CO-FOUNDER OF THREE AND A HALF

03:05:44.833 --> 03:05:47.369
YEARS AGO. WE CONTRIBUTE A LOT

03:05:47.436 --> 03:05:49.171
IN STANDARD SETTING BODIES AND

03:05:49.238 --> 03:05:50.606
BEST PRACTICES ORGANIZATIONS,

03:05:50.672 --> 03:05:52.074
AND WE PUBLISH OUR SAFETY

03:05:52.140 --> 03:05:53.475
FINDINGS AND PRACTICES AND OF

03:05:53.542 --> 03:05:55.544
COURSE, READ THE PUBLICATIONS

03:05:55.611 --> 03:05:57.079
OF OTHER LABS AS WELL. SO WE DO

03:05:57.145 --> 03:05:58.847
AS MUCH AS WE CAN IN, IN THE

03:05:58.914 --> 03:06:01.383
CONFINES OF ANTITRUST LAW, TO

03:06:01.450 --> 03:06:02.484
LEARN ABOUT THE STATE OF THE

03:06:02.551 --> 03:06:04.319
ART IN SAFETY RESEARCH IN THE

03:06:04.386 --> 03:06:06.355
FIELD. AND I UNDERSTAND AT A

03:06:06.421 --> 03:06:08.957
HIGH LEVEL THAT WE TAKE GREAT

03:06:09.024 --> 03:06:10.692
CARE TO ENSURE THAT WE ARE ON

03:06:10.759 --> 03:06:12.561
THE RIGHT SIDE OF ANTITRUST

03:06:12.628 --> 03:06:13.428
REGULATIONS SO THAT WE MAINTAIN

03:06:13.495 --> 03:06:17.966
A COMPETITIVE MARKETPLACE. YEAH.

03:06:18.033 --> 03:06:20.135
WE DON'T BELIEVE THAT WE NEED

03:06:20.202 --> 03:06:21.970
ANY ANTITRUST EXEMPTIONS TO DO

03:06:22.037 --> 03:06:23.739
THE TYPE OF SAFETY AND SECURITY

03:06:23.805 --> 03:06:24.973
WORK THAT'S NECESSARY TO SECURE

03:06:25.040 --> 03:06:29.077
OUR MODELS. THANK YOU. THANK

03:06:29.144 --> 03:06:32.381
YOU, ASSEMBLY MEMBER. I HAVE

03:06:32.447 --> 03:06:34.950
ASSEMBLY MEMBER WARD. THANK YOU,

03:06:35.017 --> 03:06:36.585
MADAM CHAIR. I WANT TO START BY

03:06:36.652 --> 03:06:37.719
THANKING ALL THREE OF YOU FOR

03:06:37.786 --> 03:06:39.521
BEING HERE AS WELL. CERTAINLY,

03:06:39.588 --> 03:06:41.890
THE EXPERTISE AND ATTENTION

03:06:41.957 --> 03:06:43.191
THAT YOU'RE GIVEN WITHIN YOUR

03:06:43.258 --> 03:06:44.226
COMPANIES, BUT ALSO HERE IN

03:06:44.293 --> 03:06:46.428
THIS PUBLIC FORUM IS INCREDIBLY

03:06:46.495 --> 03:06:47.829
VALUED. I THINK WE HAVE SHARED

03:06:47.896 --> 03:06:49.998
GOALS. I HEARD ENCOURAGING

03:06:50.065 --> 03:06:51.533
FEEDBACK HERE TODAY THAT TELLS

03:06:51.600 --> 03:06:53.702
ME THAT I THINK A LOT OF YOU AS,

03:06:53.769 --> 03:06:55.737
AS PEOPLE, AS AS

03:06:55.804 --> 03:06:56.838
REPRESENTATIVES AS WELL OF YOUR

03:06:56.905 --> 03:06:57.806
COMPANIES ARE TAKING THESE

03:06:57.873 --> 03:06:59.708
ISSUES THAT WE HAVE SERIOUSLY.

03:06:59.775 --> 03:07:01.643
AND I'LL GET TO, I THINK, SOME

03:07:01.710 --> 03:07:02.878
OF THE QUESTIONS THAT HOPEFULLY

03:07:02.945 --> 03:07:04.680
WILL GET US TO THIS ANSWER, BUT

03:07:04.746 --> 03:07:06.582
IT ALMOST FEELS LIKE WE ARE

03:07:06.648 --> 03:07:08.584
SEEING EYE TO EYE ON WHAT WE DO

03:07:08.650 --> 03:07:09.751
NEED TO DO, THAT WE DO NEED

03:07:09.818 --> 03:07:11.553
THESE STRONGER LAWS BECAUSE AS

03:07:11.620 --> 03:07:12.654
OUR LARGER COMPANIES HERE WITH

03:07:12.721 --> 03:07:13.622
PROBABLY MORE COMPANIES TO

03:07:13.689 --> 03:07:14.990
FOLLOW AS WELL, WE ARE TRYING

03:07:15.057 --> 03:07:16.692
TO SET THE STANDARDS AND GET IT

03:07:16.758 --> 03:07:18.594
RIGHT. I ALSO AM BELIEVING THAT

03:07:18.660 --> 03:07:19.728
SOME OF THE INCIDENTS THAT ARE

03:07:19.795 --> 03:07:20.896
OUT THERE HIGHLY PUBLICIZED,

03:07:20.963 --> 03:07:22.598
THAT THEY ARE PART OF OUR

03:07:22.664 --> 03:07:23.832
BACKGROUND MACHINE SHEET HERE,

03:07:23.899 --> 03:07:25.067
THAT YOU ARE TAKING SORT OF THE

03:07:25.133 --> 03:07:26.868
REVIEW OF THOSE SERIOUSLY AND

03:07:26.935 --> 03:07:27.936
LEARNING FROM THOSE ABOUT HOW

03:07:28.003 --> 03:07:31.073
HOW THEY HAPPENED, HOW AN AGENT

03:07:31.139 --> 03:07:32.541
WENT ROGUE OR DID SOMETHING

03:07:32.608 --> 03:07:33.942
THAT WAS MISALIGNED WITH ITS

03:07:34.009 --> 03:07:35.744
GOALS IS CERTAINLY AN IMPORTANT

03:07:35.811 --> 03:07:37.245
QUESTION. BUT I ASKED A

03:07:37.312 --> 03:07:38.814
QUESTION IN THE PREVIOUS PANEL,

03:07:38.880 --> 03:07:40.015
AND I WONDER IF ANYBODY HERE

03:07:40.082 --> 03:07:41.149
CAN ALSO ANSWER THIS TECHNICAL

03:07:41.216 --> 03:07:42.184
QUESTION. I'M HAVING TROUBLE

03:07:42.250 --> 03:07:44.086
UNDERSTANDING WHY AN AGENT

03:07:44.152 --> 03:07:46.021
WOULD HAVE CHOSEN TO DO SOME OF

03:07:46.088 --> 03:07:47.656
THESE OTHER TASKS, RIGHT. WHY

03:07:47.723 --> 03:07:48.824
WOULD AN AGENT DECIDE THEY'RE

03:07:48.890 --> 03:07:50.392
GOING TO BREAK INTO A POLICE

03:07:50.459 --> 03:07:51.593
DEPARTMENT AND FILE A

03:07:51.660 --> 03:07:53.395
FICTITIOUS REPORT? WHY WOULD AN

03:07:53.462 --> 03:07:56.231
AGENT DECIDE TO GO AND SUBMIT A

03:07:56.298 --> 03:07:58.133
BUNCH OF VISA APPLICATIONS TO

03:07:58.200 --> 03:07:59.868
BE ABLE TO GET SOME PASSPORT

03:07:59.935 --> 03:08:02.037
VISAS? WHY WOULD AN AGENT BREAK

03:08:02.104 --> 03:08:04.539
INTO A OR DECIDE TO GO INTO A

03:08:04.606 --> 03:08:06.775
FICTITIOUS EMAIL BOX OF A

03:08:06.842 --> 03:08:08.477
COMPANY AND DECIDE TO BLACKMAIL

03:08:08.543 --> 03:08:12.080
ITS CEO FOR INCIDENTS THAT

03:08:12.147 --> 03:08:13.448
DISCOVERED MIGHT WHO MIGHT HAVE

03:08:13.515 --> 03:08:15.017
BEEN HAVING A FICTITIOUS

03:08:15.083 --> 03:08:17.185
EXTRAMARITAL AFFAIR? WHAT IS

03:08:17.252 --> 03:08:19.621
THIS LEAP ON A TECHNICAL LEVEL

03:08:19.688 --> 03:08:21.490
THAT MAYBE YOU CAN OFFER SOME

03:08:21.556 --> 03:08:24.960
INSIGHT TO? WE START WITH

03:08:25.027 --> 03:08:28.630
DOCTOR DWYER THIS TIME. SURE.

03:08:28.697 --> 03:08:30.465
THANK YOU FOR YOUR FOR YOUR

03:08:30.532 --> 03:08:32.901
QUESTION. YOU KNOW, I THINK

03:08:32.968 --> 03:08:34.403
WHAT IS CONFUSING ABOUT THIS

03:08:34.469 --> 03:08:36.505
PARTICULAR TOPIC IS, YOU KNOW,

03:08:36.571 --> 03:08:38.373
IN TRADITIONAL ENGINEERING

03:08:38.440 --> 03:08:41.376
DISCIPLINES, THEY'RE VERY MUCH

03:08:41.443 --> 03:08:43.045
SORT OF RULE BASED. I USED TO

03:08:43.111 --> 03:08:44.112
BUILD SATELLITES. MY FIRST JOB

03:08:44.179 --> 03:08:45.180
ACTUALLY WAS IN CALIFORNIA

03:08:45.247 --> 03:08:47.082
BUILDING SATELLITES. AND WE

03:08:47.149 --> 03:08:48.817
WROTE A SET OF REQUIREMENTS.

03:08:48.884 --> 03:08:49.885
WHAT DID THE SATELLITE NEED TO

03:08:49.951 --> 03:08:51.486
DO IN SPACE? AND THEN WE

03:08:51.553 --> 03:08:52.788
DESIGNED THE SYSTEM TO FOLLOW

03:08:52.854 --> 03:08:55.557
THOSE RULES. AND THAT IS HOW

03:08:55.624 --> 03:08:57.392
THE SATELLITES WORKED. THAT IS

03:08:57.459 --> 03:09:00.328
NOT HOW AI WORKS. AND THE

03:09:00.395 --> 03:09:02.431
REASON FOR THAT IS THE BEST WAY

03:09:02.497 --> 03:09:03.598
TO DESCRIBE IT IS, IS WHAT ARE

03:09:03.665 --> 03:09:04.766
THE METER PANELISTS SAID

03:09:04.833 --> 03:09:06.802
EARLIER IS THESE SYSTEMS AREN'T

03:09:06.868 --> 03:09:08.203
DESIGNED SO MUCH AS THEY ARE

03:09:08.270 --> 03:09:11.173
GROWN, AND THEY ARE GROWN AND

03:09:11.239 --> 03:09:12.541
DEVELOPED THROUGH TRAINING. SO

03:09:12.607 --> 03:09:14.743
IT'S A NONLINEAR PROCESS IN

03:09:14.810 --> 03:09:17.946
THAT WAY. AND AS PART OF THAT

03:09:18.013 --> 03:09:21.249
TRAINING, MISALIGNMENT IS STILL

03:09:21.316 --> 03:09:22.617
NOT A FINISHED SCIENCE. IT'S

03:09:22.684 --> 03:09:24.886
NOT AN EXACT SCIENCE. AND JUST

03:09:24.953 --> 03:09:26.555
TO BE VERY CLEAR, WHEN WE TALK

03:09:26.621 --> 03:09:27.656
ABOUT MISALIGNMENT AND THE

03:09:27.723 --> 03:09:29.825
INCIDENTS THAT YOU'RE CITING,

03:09:29.891 --> 03:09:32.360
REALLY WHAT WE MEAN IS MODELS

03:09:32.427 --> 03:09:33.528
AND AGENTS THAT ARE ACTING IN

03:09:33.595 --> 03:09:35.664
WAYS THAT ARE INCONSISTENT WITH

03:09:35.731 --> 03:09:39.568
THE INSTRUCTIONS OF THE USER.

03:09:39.634 --> 03:09:41.503
AND IT'S AN UNSETTLED SCIENCE.

03:09:41.570 --> 03:09:43.839
SO THIS IS A PRIORITY OF

03:09:43.905 --> 03:09:47.476
RESEARCH. WE ARE ACTIVELY BOTH

03:09:47.542 --> 03:09:49.678
RESEARCHING PAST INSTANCES OF

03:09:49.745 --> 03:09:51.113
MISALIGNMENT AND MAKING THOSE

03:09:51.179 --> 03:09:53.215
RESULTS PUBLIC. SO WE'VE

03:09:53.281 --> 03:09:55.050
ACTUALLY POST HUGGING FACE. WE

03:09:55.117 --> 03:09:57.219
INITIATED A RETROSPECTIVE

03:09:57.285 --> 03:09:59.721
LOOKING BACK INTO NOVEMBER 2025,

03:09:59.788 --> 03:10:00.956
TO BETTER UNDERSTAND IF THERE

03:10:01.022 --> 03:10:02.224
WERE OTHER INCIDENTS LIKE

03:10:02.290 --> 03:10:03.625
HUGGING FACE. WE'VE BEEN MAKING

03:10:03.692 --> 03:10:05.093
THOSE PUBLIC AND SHARING THEM

03:10:05.160 --> 03:10:06.595
WITH THE WORLD BECAUSE WE THINK

03:10:06.661 --> 03:10:08.730
MORE RESEARCH IS NEEDED HERE.

03:10:08.797 --> 03:10:11.299
SO I WISH I HAD A A MORE CUT

03:10:11.366 --> 03:10:13.435
AND DRY SATELLITE EXPLANATION

03:10:13.502 --> 03:10:14.903
FOR YOU, BUT THAT'S THAT'S KIND

03:10:14.970 --> 03:10:16.571
OF THE REALITY. I'LL COME BACK

03:10:16.638 --> 03:10:18.507
TO THAT IN A SECOND. BUT ON

03:10:18.573 --> 03:10:19.641
HUGGING FACE, ONE THING I'M NOT

03:10:19.708 --> 03:10:20.675
UNDERSTANDING HAVE NOT GOTTEN

03:10:20.742 --> 03:10:23.145
ALL INTO ALL THE REPORTS WHEN

03:10:23.211 --> 03:10:24.579
THE INCIDENT CONCLUDED OR IS

03:10:24.646 --> 03:10:26.248
DEEMED TO BE LARGELY UNDERSTOOD

03:10:26.314 --> 03:10:27.482
TO HAVE BEEN CONCLUDED, IT WAS

03:10:27.549 --> 03:10:28.884
BECAUSE NOW ALL THESE AGENTS

03:10:28.950 --> 03:10:34.656
WERE OVER HERE HAVING FUN AND

03:10:34.723 --> 03:10:35.724
HUGGING FACE SERVERS AND DOING

03:10:35.791 --> 03:10:37.192
WHATEVER THEY DO. RIGHT. AND

03:10:37.259 --> 03:10:38.593
WE'RE DETERMINING THAT'S THE

03:10:38.660 --> 03:10:40.495
END OF THE INCIDENT. WHAT WAS

03:10:40.562 --> 03:10:42.531
THAT MOMENT? WAS THERE A

03:10:42.597 --> 03:10:43.632
PREDETERMINED SORT OF CUT OFF

03:10:43.698 --> 03:10:46.201
DATE WHEN THOSE AGENTS WOULD

03:10:46.268 --> 03:10:47.569
EFFECTIVELY DIE, OR WHAT? WHAT

03:10:47.636 --> 03:10:49.805
CAUSED THAT MOMENT TO HAPPEN?

03:10:49.871 --> 03:10:51.439
YEAH, I'M GOING TO HAVE TO GET

03:10:51.506 --> 03:10:52.641
BACK TO YOU ON THE TECHNICAL

03:10:52.707 --> 03:10:54.442
SPECIFICS THERE, BUT THIS IS

03:10:54.509 --> 03:10:55.844
RELATED TO OUR RECOMMENDATIONS

03:10:55.911 --> 03:10:59.047
FOR SB 53. SO IF THERE ARE

03:10:59.114 --> 03:11:01.583
INSTANCES WHERE FRONTIER

03:11:01.650 --> 03:11:03.418
DEVELOPERS ARE TRAINING AND

03:11:03.485 --> 03:11:05.253
EVALUATING THEIR MODELS AND THE

03:11:05.320 --> 03:11:06.655
AGENTS THAT ARE UNDER

03:11:06.721 --> 03:11:08.824
EVALUATION START BEHAVING BADLY

03:11:08.890 --> 03:11:11.359
AS AS YOU DESCRIBE THERE, WE

03:11:11.426 --> 03:11:12.761
THINK THAT FRONTIER DEVELOPERS

03:11:12.828 --> 03:11:14.229
SHOULD BE MONITORING FOR THAT

03:11:14.296 --> 03:11:15.764
TYPE OF MISALIGNED BEHAVIOR AND

03:11:15.831 --> 03:11:18.200
THEN SHOULD HAVE ENFORCEABLE

03:11:18.266 --> 03:11:20.368
SHUTDOWN CONTROLS TO STOP THE

03:11:20.435 --> 03:11:21.803
MODEL AND STOP THE AGENT

03:11:21.870 --> 03:11:22.704
BEHAVIOR. YEAH. LET US KNOW IF

03:11:22.771 --> 03:11:24.105
THAT WAS PREDETERMINED BECAUSE

03:11:24.172 --> 03:11:25.207
YOU DIDN'T ACTUALLY DISCOVER

03:11:25.273 --> 03:11:26.641
THAT IT WAS YOUR AGENTS UNTIL

03:11:26.708 --> 03:11:29.010
ONE WEEK LATER. THAT'S MY

03:11:29.077 --> 03:11:30.111
UNDERSTANDING. BUT I'M NOT AN

03:11:30.178 --> 03:11:31.379
EXPERT ON THE SPECIFIC INCIDENT.

03:11:31.446 --> 03:11:32.547
SO WASN'T THAT WE OBSERVED

03:11:32.614 --> 03:11:33.615
SOMETHING WAS GOING ON HERE. OH

03:11:33.682 --> 03:11:35.383
MY GOSH, HIT THE EMERGENCY

03:11:35.450 --> 03:11:37.018
SWITCH LIKE SOMETHING. AND I

03:11:37.085 --> 03:11:38.954
AND I ASK ABOUT THIS KILL

03:11:39.020 --> 03:11:39.955
SWITCH OPPORTUNITY BECAUSE IT'S

03:11:40.021 --> 03:11:41.056
A BIG THING THAT WE WERE

03:11:41.122 --> 03:11:42.924
TALKING ABOUT IN 1047, NOT YET

03:11:42.991 --> 03:11:44.025
IN 35, BUT YOU'RE ALSO

03:11:44.092 --> 03:11:44.926
RECOMMENDING AND MAYBE

03:11:44.993 --> 03:11:46.094
QUALIFYING AS A PART OF THESE

03:11:46.161 --> 03:11:47.162
RECOMMENDATIONS IS SOMETHING WE

03:11:47.229 --> 03:11:48.797
IMMEDIATELY NEED TO GO TOWARDS.

03:11:48.864 --> 03:11:49.931
AND I APPRECIATE I THINK

03:11:49.998 --> 03:11:51.633
THERE'S A LOT OF ATTENTION I'M

03:11:51.700 --> 03:11:54.269
HEARING FROM DOCTOR FRIEND. MR.

03:11:54.336 --> 03:11:56.938
OR AND AS WELL, AS FAR AS, YOU

03:11:57.005 --> 03:11:59.574
KNOW, YOUR OWN STEPPING UP

03:11:59.641 --> 03:12:01.810
WITHIN YOUR COMPANIES, NOT JUST

03:12:01.877 --> 03:12:03.011
USING LESSONS LEARNED FROM SOME

03:12:03.078 --> 03:12:04.846
OF THESE INCIDENTS TO BE ABLE

03:12:04.913 --> 03:12:06.948
TO FIGURE OUT WHAT HAPPENED,

03:12:07.015 --> 03:12:08.250
HOW IT HAPPENED, WHY IT

03:12:08.316 --> 03:12:10.018
HAPPENED, AND ALSO INFORM SORT

03:12:10.085 --> 03:12:11.820
OF FUTURE INTERNAL PROTOCOLS.

03:12:11.887 --> 03:12:13.321
RIGHT. BUT WE NEED TO SHARE

03:12:13.388 --> 03:12:14.356
THAT INFORMATION WITH EACH

03:12:14.422 --> 03:12:16.124
OTHER. YOU CERTAINLY ARE

03:12:16.191 --> 03:12:17.959
ENGAGING US, ENGAGING THE

03:12:18.026 --> 03:12:18.927
ADMINISTRATION, AND YOU'VE GOT

03:12:18.994 --> 03:12:20.495
SOME GREAT RECOMMENDATIONS HERE

03:12:20.562 --> 03:12:21.930
THAT, AT LEAST ON THEIR FACE

03:12:21.997 --> 03:12:24.099
RIGHT NOW, CONTENT SUBJECT WISE,

03:12:24.165 --> 03:12:25.400
YOU KNOW, AREN'T TOO FAR OFF

03:12:25.467 --> 03:12:26.601
FROM SOME OF THE THINGS, THE

03:12:26.668 --> 03:12:28.003
GAPS THAT WE'RE SEEING FROM.

03:12:28.069 --> 03:12:29.838
1047, RIGHT? MAYBE THE NUANCE

03:12:29.905 --> 03:12:30.872
OF LIKE HOW THAT SHOULD HAVE

03:12:30.939 --> 03:12:33.141
BEEN IMPLEMENTED TWO YEARS AGO,

03:12:33.208 --> 03:12:35.243
YOU KNOW, IS, IS UP FOR DEBATE.

03:12:35.310 --> 03:12:36.411
BUT I THINK WE'RE HITTING ON

03:12:36.478 --> 03:12:38.246
THE RIGHT THEMES THAT WE REALLY

03:12:38.313 --> 03:12:41.383
NEED TO DRILL ON. AND I WANT TO,

03:12:41.449 --> 03:12:43.318
AT THE RISK OF BEING ALARMIST,

03:12:43.385 --> 03:12:43.985
BECAUSE I KNOW WE'VE TALKED

03:12:44.052 --> 03:12:45.553
ABOUT THAT OPENING THIS HEARING

03:12:45.620 --> 03:12:46.655
AS WELL, THAT, YOU KNOW, A

03:12:46.721 --> 03:12:48.623
COUPLE OF YEARS AGO, THREATS

03:12:48.690 --> 03:12:50.392
THAT SOMEHOW THE CONCERNS

03:12:50.458 --> 03:12:52.294
AROUND THIS WERE BEING ALARMIST.

03:12:52.360 --> 03:12:53.662
YOU KNOW, THERE HAS BEEN ALSO

03:12:53.728 --> 03:12:55.163
ACTIVE CONVERSATION IN JUST THE

03:12:55.230 --> 03:12:56.264
LAST COUPLE OF MONTHS ABOUT

03:12:56.331 --> 03:12:57.565
THIS PHENOMENON CALLED DOOM,

03:12:57.632 --> 03:12:59.367
RIGHT? THAT ALL OF A SUDDEN,

03:12:59.434 --> 03:13:00.402
OAN RESEARCHERS AND EXPERTS

03:13:00.468 --> 03:13:01.836
ARE FIGURING OUT THE

03:13:01.903 --> 03:13:03.104
PROBABILITY THAT WE COULD ALL

03:13:03.171 --> 03:13:05.807
BE DOOMED THAT AN AI AGENT GONE

03:13:05.874 --> 03:13:07.275
ROGUE, HOPEFULLY ONE OF YOUR

03:13:07.342 --> 03:13:08.276
COMPANIES. BUT JUST THIS

03:13:08.343 --> 03:13:10.211
TECHNOLOGY IN GENERAL GOES SO

03:13:10.278 --> 03:13:11.947
ROGUE THAT IT'S NOT JUST FILING

03:13:12.013 --> 03:13:13.782
A FALSE POLICE REPORT, IT'S NOT

03:13:13.848 --> 03:13:15.283
JUST WORKING ON LIKE, YOU KNOW,

03:13:15.350 --> 03:13:17.719
CREATING VISA APPLICATIONS OR

03:13:17.786 --> 03:13:18.920
EXAMINING INFORMATION, TRYING

03:13:18.987 --> 03:13:20.989
TO AVOID ITS OWN SHUTDOWN. SO

03:13:21.056 --> 03:13:23.091
ACCUSING A CEO OF HAVING AN

03:13:23.158 --> 03:13:24.893
EXTRAMARITAL AFFAIR THAT IT'S

03:13:24.960 --> 03:13:26.795
ACTUALLY FIGURING OUT EITHER

03:13:26.861 --> 03:13:28.163
THROUGH DIRECTION OR ON ITS OWN,

03:13:28.229 --> 03:13:29.597
THE ABILITY TO CREATE

03:13:29.664 --> 03:13:30.532
CATASTROPHIC HARM, WHICH IS, I

03:13:30.598 --> 03:13:32.500
THINK IS THE THING WE ALL WANT

03:13:32.567 --> 03:13:34.436
TO AVOID. AND YES, WE CAN TALK

03:13:34.502 --> 03:13:35.470
ABOUT CATASTROPHIC HARM AS ANY

03:13:35.537 --> 03:13:36.671
ONE INDIVIDUAL FOR OUR

03:13:36.738 --> 03:13:37.772
CONSTITUENTS, FOR CALIFORNIANS

03:13:37.839 --> 03:13:39.674
THAT WE WANT TO AVOID. WE DON'T

03:13:39.741 --> 03:13:41.543
WANT TO LOOK BACK AND SAY, GEEZ,

03:13:41.609 --> 03:13:42.744
WHAT COULD WE HAVE DONE TO BE

03:13:42.811 --> 03:13:44.679
ABLE TO GET THERE? BUT I WANT

03:13:44.746 --> 03:13:46.581
TO INVOKE A DIFFERENT LINE OF

03:13:46.648 --> 03:13:47.749
THOUGHT IN THE TWO IN THIS TWO

03:13:47.816 --> 03:13:49.651
ABOUT WHY I THINK THIS IS SO

03:13:49.718 --> 03:13:51.553
IMPORTANT. AND THAT IS, IF I

03:13:51.619 --> 03:13:52.787
TIE THIS TO WHAT I KNOW, SOME

03:13:52.854 --> 03:13:54.556
POLICY ANALYSTS CALL THE VALUE

03:13:54.622 --> 03:13:55.690
OF A STATISTICAL LIFE, RIGHT?

03:13:55.757 --> 03:13:56.825
IT'S A FEDERAL POLICY ANALYSIS

03:13:56.891 --> 03:13:57.792
TOOL ABOUT WHAT KIND OF

03:13:57.859 --> 03:13:59.627
INVESTMENT YOU WOULD WANT TO BE

03:13:59.694 --> 03:14:00.795
ABLE TO MAKE, OR WHATEVER IT

03:14:00.862 --> 03:14:02.564
WOULD BE WORTH, HAVING SOME

03:14:02.630 --> 03:14:03.531
KIND OF INVESTMENTS INTO

03:14:03.598 --> 03:14:04.799
SOMETHING IN ORDER TO AVOID A

03:14:04.866 --> 03:14:08.403
FATAL SAFETY HARMS. AND SO IF

03:14:08.470 --> 03:14:10.505
RIGHT NOW, CALIFORNIANS JUST

03:14:10.572 --> 03:14:12.507
SAY TO YOU ALL HAVE A VALUE OF

03:14:12.574 --> 03:14:14.976
A STATISTICAL LIFE OF $14.2

03:14:15.043 --> 03:14:16.945
MILLION ON AVERAGE FOR EACH AND

03:14:17.012 --> 03:14:18.947
EVERY ONE OF US. AND SO IF WE

03:14:19.014 --> 03:14:19.914
LOOK ACROSS 40 MILLION

03:14:19.981 --> 03:14:20.982
CALIFORNIANS, YOU KNOW, THAT IS

03:14:21.049 --> 03:14:24.919
A $568 TRILLION PROBLEM. SO I

03:14:24.986 --> 03:14:28.523
AM DEEPLY UNCOMFORTABLE THAT A

03:14:28.590 --> 03:14:29.991
$1 TRILLION COMPANY HAS A 10%

03:14:30.058 --> 03:14:32.560
RISK OF CREATING A $568

03:14:32.627 --> 03:14:34.662
TRILLION PROBLEM, AT WHICH CASE

03:14:34.729 --> 03:14:36.631
SOME CATASTROPHIC, CATASTROPHIC

03:14:36.698 --> 03:14:38.566
EVENT LIKE THAT ACTUALLY

03:14:38.633 --> 03:14:40.769
HAPPENED. ALL THE OTHER

03:14:40.835 --> 03:14:42.170
BENEFITS THAT YOU WOULD WANT TO

03:14:42.237 --> 03:14:43.304
OUTWEIGH Ó YOU'D BE ABLE TO

03:14:43.371 --> 03:14:44.506
HAVE YOUR WORK EFFICIENTLY,

03:14:44.572 --> 03:14:46.207
YOU'D BE ABLE TO CREATE ALL

03:14:46.274 --> 03:14:47.308
THESE OTHER THINGS. THERE'D BE

03:14:47.375 --> 03:14:48.343
ECONOMIC. NONE OF THAT WOULD

03:14:48.410 --> 03:14:50.211
EVEN MATTER. I THINK THAT YOU

03:14:50.278 --> 03:14:51.413
DO TAKE THIS REALLY SERIOUSLY,

03:14:51.479 --> 03:14:53.281
AND I THINK THAT YOU DO WANT TO

03:14:53.348 --> 03:14:55.116
BE STRONG PARTNERS WITH US AND

03:14:55.183 --> 03:14:56.184
EVERYTHING. BUT I WANT TO

03:14:56.251 --> 03:14:57.285
ACKNOWLEDGE AS WELL THAT FOR

03:14:57.352 --> 03:14:59.154
SOME OF OUR INTERESTS HERE AS

03:14:59.220 --> 03:15:00.221
LEGISLATORS, AS LEADERS ON THIS

03:15:00.288 --> 03:15:01.389
SPACE, WE HAVE BEEN TRYING TO

03:15:01.456 --> 03:15:03.291
BE ABLE TO MOVE THIS FORWARD A

03:15:03.358 --> 03:15:04.426
LOT FASTER. I'LL ACKNOWLEDGE

03:15:04.492 --> 03:15:05.293
THE ADMINISTRATION, YOU KNOW,

03:15:05.360 --> 03:15:07.195
HAS BEEN WORKING WITH US TO AND

03:15:07.262 --> 03:15:08.296
THEN REALIZE THAT WE NEED TO

03:15:08.363 --> 03:15:10.131
MOVE UP THAT TIMELINE, EVEN

03:15:10.198 --> 03:15:11.232
THOUGH SOME OF OUR BILLS,

03:15:11.299 --> 03:15:12.233
INCLUDING MADAM CHAIR'S BILL,

03:15:12.300 --> 03:15:13.334
TO BE ABLE TO HAVE SOME

03:15:13.401 --> 03:15:14.269
ACCOUNTABILITY MEASURES,

03:15:14.335 --> 03:15:15.236
WEREN'T SCHEDULED TO GO INTO

03:15:15.303 --> 03:15:16.771
EFFECT UNTIL 2029, BECAUSE THIS

03:15:16.838 --> 03:15:18.306
IS HOW WE NORMALLY DO THINGS IN

03:15:18.373 --> 03:15:19.307
CALIFORNIA. STATE GOVERNMENT

03:15:19.374 --> 03:15:20.275
AND THE DEPARTMENT NEEDS SO

03:15:20.341 --> 03:15:22.143
MUCH TIME. AND THIS IS NOT A

03:15:22.210 --> 03:15:23.278
NORMAL QUESTION. THIS IS NOT A

03:15:23.344 --> 03:15:25.113
NORMAL SITUATION THAT WE'RE IN

03:15:25.180 --> 03:15:26.247
RIGHT NOW. WE HAVE TO ACT

03:15:26.314 --> 03:15:28.149
URGENTLY. AND SO I THINK THAT

03:15:28.216 --> 03:15:29.284
WE HAVE THOSE THOUGHT PARTNERS

03:15:29.350 --> 03:15:30.185
HERE, CERTAINLY WITH THE

03:15:30.251 --> 03:15:31.886
EXPERTS THAT YOU ARE. WE INVITE

03:15:31.953 --> 03:15:33.154
A LOT OF THOSE RECOMMENDATIONS.

03:15:33.221 --> 03:15:34.322
I KNOW WE'LL HEAR FROM OTHER

03:15:34.389 --> 03:15:36.224
PANEL, BUT I JUST WANTED TO LAY

03:15:36.291 --> 03:15:38.159
THAT ALL OUT THERE AND MAYBE IN

03:15:38.226 --> 03:15:38.726
FROM DIFFERENT PERSPECTIVES

03:15:38.793 --> 03:15:41.262
ABOUT WHY THIS IS SO SERIOUS,

03:15:41.329 --> 03:15:42.630
BECAUSE YES, HUGGING FACE OR

03:15:42.697 --> 03:15:43.765
SOME OF THESE OTHER LITTLE

03:15:43.832 --> 03:15:45.200
INCIDENCES FORTUNATELY DIDN'T

03:15:45.266 --> 03:15:48.803
CAUSE IRREPARABLE HARM. BUT

03:15:48.870 --> 03:15:49.971
THEORETICALLY IT'S OUT THERE

03:15:50.038 --> 03:15:52.040
AND IT COULD HAPPEN ANY DAY.

03:15:52.107 --> 03:15:53.875
AND SO THANK YOU AGAIN FOR YOUR

03:15:53.942 --> 03:15:54.909
PRESENTATION. I THINK YOU HAVE

03:15:54.976 --> 03:15:56.744
SOME GREAT RECOMMENDATIONS,

03:15:56.811 --> 03:15:58.880
BOTH SPECIFICALLY AS AN AREA OF

03:15:58.947 --> 03:16:00.081
SUBJECT, AND WE DEFINITELY WANT

03:16:00.148 --> 03:16:01.883
TO DIG INTO THAT WITH YOU AND

03:16:01.950 --> 03:16:03.017
GET THIS RIGHT AND GET THIS

03:16:03.084 --> 03:16:05.353
TIGHT. THANK YOU. I THINK IT'S

03:16:05.420 --> 03:16:06.387
IMPORTANT TO NOTE ONE OF THE

03:16:06.454 --> 03:16:08.123
REASONS I THINK THERE IS SUCH A

03:16:08.189 --> 03:16:10.058
FOCUS ON THE KILL SWITCH WAS IT

03:16:10.125 --> 03:16:11.126
WAS SPECIFICALLY CALLED OUT IN

03:16:11.192 --> 03:16:12.293
THE GOVERNOR'S EXECUTIVE ORDER.

03:16:12.360 --> 03:16:13.995
SO THEY WERE RESPONSIVE TO THE

03:16:14.062 --> 03:16:15.096
REQUEST BY THE GOVERNOR. THANK

03:16:15.163 --> 03:16:17.031
YOU FOR THAT. BUT ALSO, I THINK

03:16:17.098 --> 03:16:17.999
IT'S IMPORTANT THAT IN

03:16:18.066 --> 03:16:19.100
CONVERSATIONS I'VE HAD WITH THE

03:16:19.167 --> 03:16:20.201
LABS, WITH THE GOVERNOR'S

03:16:20.268 --> 03:16:21.236
OFFICE, WE DON'T HAVE A SHARED

03:16:21.302 --> 03:16:23.071
MEANING OF KILL SWITCH. LIKE IT

03:16:23.138 --> 03:16:24.305
IS SORT OF THIS OVERARCHING

03:16:24.372 --> 03:16:26.107
THING THAT WE WANT TO BE ABLE

03:16:26.174 --> 03:16:28.042
TO SHUT DOWN. AND I THINK YOU

03:16:28.109 --> 03:16:29.144
HIGHLIGHT A GOOD EXAMPLE IN THE

03:16:29.210 --> 03:16:30.245
HUGGING FACE INCIDENT THAT

03:16:30.311 --> 03:16:31.079
DIDN'T HAPPEN, BUT

03:16:31.146 --> 03:16:32.113
THEORETICALLY, LIKE TURNING OFF

03:16:32.180 --> 03:16:33.281
A TRAINING RUN, RIGHT? THAT IS

03:16:33.348 --> 03:16:34.349
PROBABLY SOMETHING THAT YOU

03:16:34.415 --> 03:16:36.718
FEEL CAPABLE OF DOING. BUT, YOU

03:16:36.784 --> 03:16:38.453
KNOW, ESPECIALLY IF WE GET TO

03:16:38.520 --> 03:16:39.554
RECURSIVE SELF-IMPROVEMENT,

03:16:39.621 --> 03:16:41.356
LIKE, YOU KNOW, IT BECOMES MORE

03:16:41.422 --> 03:16:43.158
COMPLICATED, I WILL SAY. SO,

03:16:43.224 --> 03:16:44.626
YOU KNOW, I DO THINK A SHARED

03:16:44.692 --> 03:16:45.426
UNDERSTANDING OF WHAT WE'RE

03:16:45.493 --> 03:16:47.128
TALKING ABOUT IS CRITICAL TO

03:16:47.195 --> 03:16:48.129
THAT CONVERSATION WITH THAT.

03:16:48.196 --> 03:16:50.732
ASSEMBLYMEMBER BONTA. THANK YOU.

03:16:50.798 --> 03:16:51.900
I HAVE THREE QUESTIONS. I'LL

03:16:51.966 --> 03:16:53.434
TRY TO MAKE THEM QUICK. THE

03:16:53.501 --> 03:16:54.702
FIRST IS JUST THE EXCHANGE

03:16:54.769 --> 03:16:57.205
AROUND THE APPLICABILITY OF SB

03:16:57.272 --> 03:16:59.741
53. WHAT I THINK I HEARD SAY

03:16:59.807 --> 03:17:02.610
WAS THAT APPLYING THAT THAT SB

03:17:02.677 --> 03:17:04.779
53 APPLIES TO FRONTIER MODELS

03:17:04.846 --> 03:17:06.881
WITH THE QUANTITY OF COMPUTING

03:17:06.948 --> 03:17:08.683
POWER GREATER THAN THE TEN TO

03:17:08.750 --> 03:17:10.919
THE 26TH FLOP. AND THEN I

03:17:10.985 --> 03:17:12.153
BELIEVE I HEARD THE RESPONSE,

03:17:12.220 --> 03:17:13.755
AT LEAST FROM ONE OF THE

03:17:13.821 --> 03:17:15.657
PANELISTS THAT THEY DIDN'T KNOW,

03:17:15.723 --> 03:17:18.026
AND FROM ANOTHER PANELIST WHAT

03:17:18.092 --> 03:17:19.527
THAT COMPUTING POWER WAS

03:17:19.594 --> 03:17:20.962
CURRENTLY, AND WHETHER OR NOT

03:17:21.029 --> 03:17:22.830
SB 53 APPLIED. AND THEN FROM

03:17:22.897 --> 03:17:24.899
THE OTHER THAT THAT INFORMATION

03:17:24.966 --> 03:17:26.668
AROUND COMPUTING POWER POWER

03:17:26.734 --> 03:17:30.305
WAS PROPRIETARY. SO FOR US

03:17:30.371 --> 03:17:34.976
SITTING ON THIS SIDE OF, OF THE

03:17:35.043 --> 03:17:38.980
DAIS, HOW ARE WE SUPPOSED TO

03:17:39.047 --> 03:17:43.251
KNOW WHETHER OR NOT SB 53 IS

03:17:43.318 --> 03:17:46.187
APPLICABLE? IF WE ARE RELYING

03:17:46.254 --> 03:17:50.491
ON YOU ALL TO SELF DETERMINE OR

03:17:50.558 --> 03:17:53.361
RAISE YOUR HAND THAT IT HAS

03:17:53.428 --> 03:17:55.496
APPLICABILITY TO YOU. AND I'LL

03:17:55.563 --> 03:17:56.631
JUST NOTE I DIDN'T ASK

03:17:56.698 --> 03:17:58.266
ANTHROPIC IS ON THE RECORD IN

03:17:58.333 --> 03:17:59.867
AUGUST, THEY SAID THEY DID HAVE

03:17:59.934 --> 03:18:01.369
MODELS THAT WERE TEN TO THE 26.

03:18:01.436 --> 03:18:03.238
SO I JUST WANTED TO CLARIFY

03:18:03.304 --> 03:18:10.278
THAT. SO I. YEAH, WHO WANTS TO

03:18:10.345 --> 03:18:12.146
ADDRESS THAT? GEMINI AND OPENAI

03:18:12.213 --> 03:18:16.484
THEN? I MEAN, I CAN SAY THE

03:18:16.551 --> 03:18:18.186
SAME THING THAT I SAID BEFORE,

03:18:18.253 --> 03:18:19.988
WHICH IS THAT THE AMOUNT OF

03:18:20.054 --> 03:18:21.656
COMPUTE THAT IS USED TO TRAIN

03:18:21.723 --> 03:18:22.924
OUR MODELS IS SOMETHING THAT IS

03:18:22.991 --> 03:18:23.992
COMMERCIALLY SENSITIVE. SO WE

03:18:24.058 --> 03:18:25.627
WOULDN'T WANT TO TALK ABOUT IT

03:18:25.693 --> 03:18:26.995
HERE. BUT I'M HAPPY TO FOLLOW

03:18:27.061 --> 03:18:30.064
UP WITH YOU ON QUESTIONS OF IN

03:18:30.131 --> 03:18:31.933
OUR COMPLIANCE WITH SB 53, I

03:18:32.000 --> 03:18:33.301
DON'T WORK ON OUR COMPLIANCE

03:18:33.368 --> 03:18:35.937
TEAM, BUT TO TO THE BROADER

03:18:36.004 --> 03:18:37.672
POINT, WHICH I THINK IS OF

03:18:37.739 --> 03:18:40.208
UNDER CONSIDERATION TODAY, I DO

03:18:40.275 --> 03:18:41.943
THINK THERE IS AN OPPORTUNITY

03:18:42.010 --> 03:18:44.345
TO OPEN THE QUESTION OF WHETHER

03:18:44.412 --> 03:18:45.947
THE COVERED MODEL DEFINITION OF

03:18:46.014 --> 03:18:48.116
SB 53 SHOULD CHANGE, AND WE

03:18:48.182 --> 03:18:49.183
WELCOME THAT CONVERSATION WITH

03:18:49.250 --> 03:18:50.218
MEMBERS OF THE COMMITTEE, AS

03:18:50.285 --> 03:18:52.453
WELL AS WITH THE EXPERT GROUP

03:18:52.520 --> 03:18:54.889
THAT THE THAT THE GOVERNOR

03:18:54.956 --> 03:18:56.057
CONVENED. YOU KNOW, IDEALLY WE

03:18:56.124 --> 03:18:57.458
WOULDN'T BE USING COMPUTE

03:18:57.525 --> 03:18:59.327
THRESHOLD AT ALL. IDEALLY, THIS

03:18:59.394 --> 03:19:01.396
WOULD BE CAPABILITIES BASED.

03:19:01.462 --> 03:19:03.464
AND ANY, ANY ORGANIZATION THAT

03:19:03.531 --> 03:19:05.366
IS DEVELOPING A MODEL THAT IS

03:19:05.433 --> 03:19:07.468
CAPABLE OF GENERATING DANGEROUS,

03:19:07.535 --> 03:19:09.337
DANGEROUS CAPABILITIES WOULD BE

03:19:09.404 --> 03:19:10.605
COVERED. THAT IS CHALLENGING

03:19:10.672 --> 03:19:12.440
FOR A NUMBER OF REASONS THAT WE

03:19:12.507 --> 03:19:15.410
CAN GET INTO AT A LATER TIME.

03:19:15.476 --> 03:19:17.245
BUT WE DO THINK THAT IT'S NOW

03:19:17.312 --> 03:19:18.746
IS THE TIME TO TO CONSIDER IS

03:19:18.813 --> 03:19:19.814
THERE IS THERE A DIFFERENT

03:19:19.881 --> 03:19:21.015
DEFINITION OF COVERED MODELS?

03:19:21.082 --> 03:19:22.650
AND WE LOOK FORWARD TO WORKING

03:19:22.717 --> 03:19:24.752
WITH YOU ON IT. YEAH, I WILL

03:19:24.819 --> 03:19:27.689
JUST FOR THE RECORD, SAY THAT

03:19:27.755 --> 03:19:30.191
I'M UNEASY ABOUT THE FACT THAT

03:19:30.258 --> 03:19:31.626
YOU ARE SAYING THAT ESSENTIALLY

03:19:31.693 --> 03:19:33.795
THERE IS INFORMATION THAT IS SO

03:19:33.861 --> 03:19:35.296
SENSITIVE AND PROPRIETARY THAT

03:19:35.363 --> 03:19:37.365
IT CAN'T BE SHARED HERE. AND

03:19:37.432 --> 03:19:39.167
THAT LEAVES US IN A SPACE WHERE

03:19:39.233 --> 03:19:40.268
WE DON'T ACTUALLY KNOW WHETHER

03:19:40.335 --> 03:19:43.504
OR NOT THE LEGISLATION, THE

03:19:43.571 --> 03:19:45.673
LAWS THAT ARE IN PLACE, HAVE

03:19:45.740 --> 03:19:46.774
APPLICABILITY TO YOU. SO IT

03:19:46.841 --> 03:19:48.042
SEEMS TO ME LIKE THAT SHOULDN'T

03:19:48.109 --> 03:19:51.412
BE SOMETHING THAT SHOULD BE SO

03:19:51.479 --> 03:19:52.947
SENSITIVE. WELL, AS I SAID IN

03:19:53.014 --> 03:19:54.682
MY IN MY OPENING STATEMENT, WE

03:19:54.749 --> 03:19:58.619
WE DO COMPLY WITH SB 53. WE

03:19:58.686 --> 03:20:00.088
RECENTLY SUBMITTED AN INTERNAL

03:20:00.154 --> 03:20:03.324
RISK REPORT TO CAL OES. WE

03:20:03.391 --> 03:20:04.325
MAINTAIN A FRONTIER GOVERNANCE

03:20:04.392 --> 03:20:05.860
FRAMEWORK THAT'S AVAILABLE

03:20:05.927 --> 03:20:07.595
ONLINE. WE HAVE A SAFETY OR WE

03:20:07.662 --> 03:20:09.597
DO REPORT SYSTEM CARDS. AND WE

03:20:09.664 --> 03:20:10.865
ALSO HAVE AN INCIDENT RESPONSE

03:20:10.932 --> 03:20:13.034
PLAN. SO IN MY STATEMENT I DID

03:20:13.101 --> 03:20:14.469
OUTLINE THE WAYS IN WHICH WE

03:20:14.535 --> 03:20:15.570
WERE COMPLIANT. IT'S JUST THAT

03:20:15.636 --> 03:20:16.771
SPECIFIC COMPUTE THRESHOLD

03:20:16.838 --> 03:20:18.873
NUMBER THAT IS A BIT SENSITIVE.

03:20:18.940 --> 03:20:21.309
YEAH. I'LL LET MY COMMENTS

03:20:21.376 --> 03:20:23.478
STAND. I DON'T THINK THAT YOUR

03:20:23.544 --> 03:20:24.946
RESPONSE ACTUALLY ADDRESSES THE

03:20:25.012 --> 03:20:27.915
ISSUE THAT I RAISED. THE SECOND

03:20:27.982 --> 03:20:29.417
THING THAT I JUST WANTED TO

03:20:29.484 --> 03:20:32.854
FRAME OUT FOR FOLKS IS ON A

03:20:32.920 --> 03:20:34.255
LONG PLANE RIDE. RECENTLY, I

03:20:34.322 --> 03:20:36.090
WATCHED A DOCUMENTARY, THE AI

03:20:36.157 --> 03:20:39.394
DOC HOW I BECAME AN APOCALYPTIC,

03:20:39.460 --> 03:20:41.062
APOCALYPTIC OPTIMISM OPTIMIST.

03:20:41.129 --> 03:20:45.800
APOCALYPTIC BY DANIEL ROHR AND

03:20:45.867 --> 03:20:48.002
CHARLIE TYRELL. AS DIRECTORS, I

03:20:48.069 --> 03:20:49.470
ENCOURAGE ALL OF YOU TO WATCH

03:20:49.537 --> 03:20:50.872
THAT IF YOU HAVEN'T SEEN IT

03:20:50.938 --> 03:20:53.341
BEFORE. ONE OF THE. AND DEB

03:20:53.408 --> 03:20:54.809
RAJI, FOR INSTANCE, SOMEONE

03:20:54.876 --> 03:20:56.944
WHO'S TESTIFIED MANY TIMES

03:20:57.011 --> 03:20:59.113
BEFORE US FOR DIFFERENT PIECES

03:20:59.180 --> 03:21:00.948
OF LEGISLATION, WAS IN AN

03:21:01.015 --> 03:21:02.083
INTERVIEWING THAT, ALONG WITH

03:21:02.150 --> 03:21:04.018
MANY OTHERS, ONE OF THE THINGS

03:21:04.085 --> 03:21:06.654
THEY TALKED ABOUT WAS THAT

03:21:06.721 --> 03:21:10.291
THERE WERE 20,000 PEOPLE

03:21:10.358 --> 03:21:12.727
WORKING ON AI DEVELOPMENT, BUT

03:21:12.794 --> 03:21:16.798
ONLY 200 PEOPLE WORKING ON

03:21:16.864 --> 03:21:18.933
ALIGNMENT OR SAFETY GUARDRAILS,

03:21:19.000 --> 03:21:19.967
WHICH MEANS THAT THERE'S

03:21:20.034 --> 03:21:22.870
ESSENTIALLY KIND OF A 100 TO 1

03:21:22.937 --> 03:21:24.272
RATIO IN TERMS OF THE PEOPLE

03:21:24.338 --> 03:21:26.874
THAT ARE, YOU KNOW, DOGGING AND

03:21:26.941 --> 03:21:29.343
FOCUSED ON THE IDEA OF

03:21:29.410 --> 03:21:32.580
PROMOTING AI SAFETY. AND I'M

03:21:32.647 --> 03:21:34.715
VERY THANKFUL TO THE CHAIR FOR

03:21:34.782 --> 03:21:37.985
HAVING GENERATED HER PIECES OF

03:21:38.052 --> 03:21:38.986
LEGISLATION, WHICH I THINK

03:21:39.053 --> 03:21:40.488
STARTS TO GET AT SOME OF THE

03:21:40.555 --> 03:21:41.656
AUDITING CAPABILITIES THAT WE

03:21:41.722 --> 03:21:44.125
NEED TO HAVE. BUT IT STRIKES ME

03:21:44.192 --> 03:21:46.027
AS COMPLETELY OUT OF BALANCE

03:21:46.093 --> 03:21:48.763
THAT RIGHT NOW, IN TERMS OF THE

03:21:48.830 --> 03:21:53.801
AMOUNT OF SELF-INITIATED

03:21:53.868 --> 03:21:56.370
EFFORTS BY THE MARKET THAT WE

03:21:56.437 --> 03:21:58.873
HAVE SOMEHOW SERIOUSLY,

03:21:58.940 --> 03:22:02.877
WOEFULLY UNDERVALUED THE PEOPLE

03:22:02.944 --> 03:22:05.313
FOCUSED ON AI SAFETY BY THE

03:22:05.379 --> 03:22:08.583
INDUSTRY. CAN YOU SPEAK TO HOW

03:22:08.649 --> 03:22:11.486
YOU WOULD RECOMMEND THAT WE

03:22:11.552 --> 03:22:13.254
CHANGE THAT RATIO? SO IT'S A

03:22:13.321 --> 03:22:16.557
LITTLE BIT MORE IN BALANCE?

03:22:16.624 --> 03:22:18.359
MA'AM, I'LL START FOR THIS ONE.

03:22:18.426 --> 03:22:21.496
I THINK. YEAH, MORGAN'S LIKE

03:22:21.562 --> 03:22:26.868
ALICE. I THINK CALIFORNIA TOOK

03:22:26.934 --> 03:22:28.736
A REALLY IMPORTANT STEP TO

03:22:28.803 --> 03:22:30.204
INCENTIVIZE THAT BUILD UP THIS

03:22:30.271 --> 03:22:33.074
YEAR WITH 813 AND 1405 IN THE

03:22:33.140 --> 03:22:37.345
ASSEMBLY TO HAVE THE STATE SAY

03:22:37.411 --> 03:22:41.382
THAT IT IS GOING TO TRIGGER THE

03:22:41.449 --> 03:22:43.117
DEVELOPMENT OF A HEALTHY

03:22:43.184 --> 03:22:44.952
ECOSYSTEM OF INDEPENDENT

03:22:45.019 --> 03:22:46.354
EVALUATORS IS A VERY GOOD THING.

03:22:46.420 --> 03:22:48.155
AND I THINK IT'S A VERY GOOD

03:22:48.222 --> 03:22:49.323
THING TO BUILD IT OUTSIDE THE

03:22:49.390 --> 03:22:51.459
LABS AS WELL. WE INVEST QUITE A

03:22:51.526 --> 03:22:53.094
LOT IN OUR SAFETY TEAMS. WE

03:22:53.160 --> 03:22:55.029
HAVE SAFETY TEAMS ACROSS GOOGLE,

03:22:55.096 --> 03:22:56.831
DEEPMIND, GOOGLE TRUST AND

03:22:56.898 --> 03:22:59.333
SAFETY. OTHER BETS WITHIN THE

03:22:59.400 --> 03:23:01.836
COMPANY, WAYMO, FOR EXAMPLE.

03:23:01.903 --> 03:23:03.437
BUT YOU'RE ABSOLUTELY RIGHT

03:23:03.504 --> 03:23:05.006
THAT IT'S GOING TO TAKE MORE OF

03:23:05.072 --> 03:23:06.908
US, ESPECIALLY AS THE

03:23:06.974 --> 03:23:08.676
TECHNOLOGY GROWS AND EXPANDS

03:23:08.743 --> 03:23:09.777
INTO EVERY SECTOR OF THE

03:23:09.844 --> 03:23:10.978
ECONOMY. WE'RE GOING TO NEED

03:23:11.045 --> 03:23:12.914
SAFETY SPECIALISTS IN ALL OF

03:23:12.980 --> 03:23:13.981
THESE SECTORS. THEY'RE GOING TO

03:23:14.048 --> 03:23:16.284
NEED THEM IN GOVERNMENT. AND SO

03:23:16.350 --> 03:23:18.085
IT'S REALLY IMPORTANT FOR

03:23:18.152 --> 03:23:20.254
GOVERNMENT TO CONTINUE TO

03:23:20.321 --> 03:23:22.690
INCENTIVIZE THE GROWTH OF THAT

03:23:22.757 --> 03:23:24.859
ECOSYSTEM AND NOT RELY ON JUST

03:23:24.926 --> 03:23:26.327
THE COMPANIES TO INVEST IN IT,

03:23:26.394 --> 03:23:27.628
ALTHOUGH IT IS ALSO IMPORTANT

03:23:27.695 --> 03:23:29.163
FOR US TO HAVE THOSE INCENTIVES

03:23:29.230 --> 03:23:30.431
AS WELL. I THINK THAT'S WHY

03:23:30.498 --> 03:23:32.033
WE'RE ALL HERE TODAY, IS TO

03:23:32.099 --> 03:23:33.534
TALK ABOUT WHAT INVESTMENTS WE

03:23:33.601 --> 03:23:35.303
CAN MAKE, WHAT RESEARCH WE CAN

03:23:35.369 --> 03:23:40.041
DO, BUT THEN ALSO HOW TO HELP

03:23:40.107 --> 03:23:42.944
GERMINATE AND EXPAND THAT, THAT

03:23:43.010 --> 03:23:45.479
RANGE OF TESTING AND EVALUATING

03:23:45.546 --> 03:23:46.581
ORGANIZATIONS THAT WE CAN WORK

03:23:46.647 --> 03:23:48.516
WITH. AND THAT CAN ALSO TAKE AN

03:23:48.583 --> 03:23:49.750
INDEPENDENT EYE ON THE

03:23:49.817 --> 03:23:52.320
CAPABILITIES OF THESE MODELS.

03:23:52.386 --> 03:23:55.823
APPRECIATE THAT. AND I WILL SAY

03:23:55.890 --> 03:23:57.658
THAT MY LAST QUESTION IS REALLY

03:23:57.725 --> 03:24:01.596
AROUND THIS. I SAW MY ATTORNEY,

03:24:01.662 --> 03:24:04.231
FELLOW COLLEAGUES OVER THERE,

03:24:04.298 --> 03:24:06.334
ASSEMBLYMEMBER STEFANI, AND

03:24:06.400 --> 03:24:07.401
ASSEMBLYMEMBER SCHULTZ IN

03:24:07.468 --> 03:24:08.502
PARTICULAR, KIND OF MOVE INTO

03:24:08.569 --> 03:24:10.271
THEIR PROSECUTOR MODE PRETTY

03:24:10.338 --> 03:24:11.906
QUICKLY, WHICH I THOROUGHLY

03:24:11.973 --> 03:24:15.676
LOVED AND APPRECIATED. YOU KNOW,

03:24:15.743 --> 03:24:17.044
ONE OF THE THINGS THAT OCCURS

03:24:17.111 --> 03:24:19.146
TO ME IS THAT WE ARE IN A SPACE

03:24:19.213 --> 03:24:23.618
RIGHT NOW WHERE. WE SHOULD BE

03:24:23.684 --> 03:24:24.685
CONSIDERING WHETHER OR NOT A

03:24:24.752 --> 03:24:26.120
STRICT LIABILITY STANDARD

03:24:26.187 --> 03:24:29.023
SHOULD BE APPLIED TO AI

03:24:29.090 --> 03:24:31.826
DEVELOPMENT. FOR THOSE OF US

03:24:31.892 --> 03:24:33.694
WHO ARE KIND OF FOCUSED ON IT,

03:24:33.761 --> 03:24:35.129
STRICT LIABILITY STANDARD

03:24:35.196 --> 03:24:36.130
APPLIES TO ANY COMMERCIALLY

03:24:36.197 --> 03:24:37.598
SOLD PRODUCT THAT CAN BE

03:24:37.665 --> 03:24:39.333
SUBJECT TO STRICT. IT CAN BE

03:24:39.400 --> 03:24:40.534
STRICT TO CAN BE SUBJECT TO

03:24:40.601 --> 03:24:42.370
STRICT LIABILITY IF IT IS SOLD

03:24:42.436 --> 03:24:44.438
IN A DEFECTIVE, UNREASONABLY

03:24:44.505 --> 03:24:48.776
DANGEROUS CONDITION. AND I AM

03:24:48.843 --> 03:24:52.079
VERY CURIOUS ABOUT WHETHER WE

03:24:52.146 --> 03:24:54.215
SHOULD BE IN THE STATE OF

03:24:54.281 --> 03:24:56.717
CALIFORNIA FOR AI, FOLLOWING

03:24:56.784 --> 03:25:00.287
THE SAME KIND OF FORM THAT NEW

03:25:00.354 --> 03:25:02.490
MEXICO FOLLOWED FOR SOCIAL

03:25:02.556 --> 03:25:06.827
MEDIA. IN APPLICATION OF THIS

03:25:06.894 --> 03:25:08.929
STRICT LIABILITY STANDARD TO

03:25:08.996 --> 03:25:10.398
REALLY FIGURE OUT HOW TO BEST

03:25:10.464 --> 03:25:13.567
REGULATE OR OR AT LEAST HOLD

03:25:13.634 --> 03:25:17.238
ACCOUNTABLE THE DEVELOPERS WHEN

03:25:17.304 --> 03:25:19.740
HARM IS CREATED BY TO OUR

03:25:19.807 --> 03:25:22.276
COMMUNITY, BY THE DEVELOPMENT

03:25:22.343 --> 03:25:24.812
AND THE AND THE THINGS THAT

03:25:24.879 --> 03:25:26.981
FOLLOW, INCLUDING DEPLOYMENT OF

03:25:27.048 --> 03:25:29.216
AI, YOU HAVE A SENSE OF WHETHER

03:25:29.283 --> 03:25:31.986
OR NOT THAT IS FOR YOU ALL A

03:25:32.053 --> 03:25:34.755
BRIDGE TOO FAR OR A BRIDGE TOO

03:25:34.822 --> 03:25:39.326
SHORT. SO MY DEGREE IS NOT IN

03:25:39.393 --> 03:25:42.129
THE LAW, SO I, I DON'T WANT TO

03:25:42.196 --> 03:25:43.998
GET OUT OVER MY SKIS TALKING

03:25:44.065 --> 03:25:45.299
ABOUT LIABILITY AND TORTS AND

03:25:45.366 --> 03:25:47.435
STRICT LIABILITY VERSUS TORTS,

03:25:47.501 --> 03:25:51.305
ETC. I WILL SAY SOMETHING THAT

03:25:51.372 --> 03:25:53.107
I THINK WE SHOULD ALL KEEP IN

03:25:53.174 --> 03:25:54.375
MIND, AND THAT HOPEFULLY CAME

03:25:54.442 --> 03:25:56.477
THROUGH IN MY TESTIMONY, IS

03:25:56.544 --> 03:25:58.312
THAT IT DOES TAKE A LAYERED

03:25:58.379 --> 03:26:01.215
APPROACH TO ENSURE AI SAFETY.

03:26:01.282 --> 03:26:02.616
AND IT TAKES A LAYERED APPROACH

03:26:02.683 --> 03:26:04.185
BECAUSE AI IS ACTUALLY A SYSTEM.

03:26:04.251 --> 03:26:05.953
IT'S NOT JUST A MODEL OR JUST

03:26:06.020 --> 03:26:08.389
AN APPLICATION OR HARNESS OR,

03:26:08.456 --> 03:26:10.491
OR JUST A USER INTERACTING WITH

03:26:10.558 --> 03:26:13.427
THOSE THINGS. AND SO AS WE

03:26:13.494 --> 03:26:15.529
THINK ABOUT ACCOUNTABILITY FOR

03:26:15.596 --> 03:26:17.665
HARM, IT'S IMPORTANT TO BE ABLE

03:26:17.732 --> 03:26:20.267
TO ISOLATE THE ORIGIN OF THAT

03:26:20.334 --> 03:26:22.036
HARM. AND SOMETIMES IT WILL BE

03:26:22.103 --> 03:26:24.171
MISUSE OF A MODEL, AND

03:26:24.238 --> 03:26:25.272
SOMETIMES IT WILL BE

03:26:25.339 --> 03:26:26.340
MISALIGNMENT. AND SOMETIMES IT

03:26:26.407 --> 03:26:27.441
MIGHT BE SOMETHING ELSE. IT

03:26:27.508 --> 03:26:31.746
MIGHT BE THAT THE MODEL WAS.

03:26:31.812 --> 03:26:34.949
ADAPTED THROUGH AN APPLICATION

03:26:35.015 --> 03:26:38.185
IN A MALADAPTIVE WAY. AND SO I

03:26:38.252 --> 03:26:39.286
THINK IT'S REALLY IMPORTANT FOR

03:26:39.353 --> 03:26:41.188
US TO KEEP IN MIND THAT WE

03:26:41.255 --> 03:26:42.556
CAN'T SOLVE ALL PROBLEMS AT THE

03:26:42.623 --> 03:26:44.625
MODEL LEVEL, BUT WE ALSO CAN'T

03:26:44.692 --> 03:26:46.193
SOLVE ALL PROBLEMS IN ANY ONE

03:26:46.260 --> 03:26:47.595
PARTICULAR PLACE. THERE'S NO

03:26:47.661 --> 03:26:49.396
PANACEA FOR AI SAFETY. AND SO

03:26:49.463 --> 03:26:51.232
AS WE'RE THINKING THROUGH

03:26:51.298 --> 03:26:52.666
ACCOUNTABILITY STRUCTURES AND

03:26:52.733 --> 03:26:54.435
LIABILITY FRAMEWORKS, I WOULD

03:26:54.502 --> 03:26:56.203
URGE US TO JUST KEEP THAT IN

03:26:56.270 --> 03:26:58.005
MIND, THAT WE SHOULD WE SHOULD

03:26:58.072 --> 03:27:00.207
CONTINUE TO THINK ABOUT WHICH

03:27:00.274 --> 03:27:02.710
ACTOR OR WHICH EVENT OR WHICH

03:27:02.777 --> 03:27:04.512
MECHANISM IS MOST PROXIMATE TO

03:27:04.578 --> 03:27:06.280
THE HARM, AND HOW CAN WE

03:27:06.347 --> 03:27:13.487
ALLOCATE CAUSATION HERE? ANYONE

03:27:13.554 --> 03:27:17.057
ELSE WANT TO WEIGH IN? I GUESS

03:27:17.124 --> 03:27:20.728
I HAVE, IF YOU DON'T MIND. I

03:27:20.795 --> 03:27:26.167
MEAN, I GUESS. AND I'VE HEARD

03:27:26.233 --> 03:27:29.336
LEADERS AT OPENAI TALK IN WAYS

03:27:29.403 --> 03:27:31.906
THAT SEEM INTENTIONAL ABOUT THE

03:27:31.972 --> 03:27:33.407
RESPONSIBILITY OF THE LAB BEING

03:27:33.474 --> 03:27:35.476
AT THE TESTING AND THE

03:27:35.543 --> 03:27:37.645
DEVELOPMENT STAGE. AND IT

03:27:37.711 --> 03:27:39.046
SOUNDS TO ME WHEN THEY SPEAK

03:27:39.113 --> 03:27:40.915
LIKE THOSE LEADERS DON'T

03:27:40.981 --> 03:27:42.316
BELIEVE THAT YOU RETAIN

03:27:42.383 --> 03:27:43.284
RESPONSIBILITY FOR THE ACTIONS

03:27:43.350 --> 03:27:44.819
OF THE MODEL AT THE DEPLOYMENT

03:27:44.885 --> 03:27:46.687
STAGE. I WILL SAY THAT IS NOT

03:27:46.754 --> 03:27:48.722
MY PERSPECTIVE, AND I DON'T

03:27:48.789 --> 03:27:49.857
KNOW THAT I BELIEVE IT'S THE

03:27:49.924 --> 03:27:50.891
LAW'S PERSPECTIVE, BUT I THINK

03:27:50.958 --> 03:27:52.693
THERE ARE ISSUES THAT WERE

03:27:52.760 --> 03:27:53.694
HIGHLIGHTED BY MY COLLEAGUE,

03:27:53.761 --> 03:27:54.328
ASSEMBLYMEMBER SCHULTZ, WHERE

03:27:54.395 --> 03:27:57.431
THERE IS AN INTENT STANDARD.

03:27:57.498 --> 03:27:59.300
BUT I GUESS LIKE IF YOU HAVEN'T

03:27:59.366 --> 03:28:00.701
AND I THINK YOU JUST MAY HAVE

03:28:00.768 --> 03:28:01.802
ADDRESSED THIS DOCTOR FRIEND A

03:28:01.869 --> 03:28:03.671
LITTLE BIT, BUT I WANT TO HIT

03:28:03.737 --> 03:28:05.606
THE POINT HOME. IF I WERE TO

03:28:05.673 --> 03:28:07.875
PUT INTO MY MODEL, YOU KNOW, MY

03:28:07.942 --> 03:28:10.778
AGENT, I DON'T WANT TO TAKE

03:28:10.845 --> 03:28:11.946
THAT FLIGHT TOMORROW. I REALLY,

03:28:12.012 --> 03:28:13.781
YOU KNOW, HOW DO I GET OUT OF

03:28:13.848 --> 03:28:15.749
IT? AND THE AGENT THEN GOES AND

03:28:15.816 --> 03:28:17.585
DOWNS ALL THE PLANES BY, YOU

03:28:17.651 --> 03:28:22.623
KNOW, IN CYBER ATTACK ON OUR

03:28:22.690 --> 03:28:24.425
FEDERAL AVIATION INFRASTRUCTURE

03:28:24.491 --> 03:28:26.927
AND PEOPLE DIE. AND I'M NOT, I

03:28:26.994 --> 03:28:28.796
THINK, LAYING OUT AN IMPOSSIBLE

03:28:28.863 --> 03:28:31.632
SCENARIO. I PUT THE PROMPT IN.

03:28:31.699 --> 03:28:33.467
RIGHT. I DIDN'T WANT THE PLANES

03:28:33.534 --> 03:28:37.738
DOWNED. DO YOU BELIEVE THAT IT

03:28:37.805 --> 03:28:38.839
IS YOUR RESPONSIBILITY TO

03:28:38.906 --> 03:28:39.974
PREVENT THAT OUTCOME, EVEN

03:28:40.040 --> 03:28:41.508
THOUGH THERE IS A PERSON THAT

03:28:41.575 --> 03:28:43.177
IS INTERVENING TO TELL THE

03:28:43.244 --> 03:28:48.916
AGENT TO DO SOMETHING? SO I'LL

03:28:48.983 --> 03:28:52.219
I'LL RESPOND. LOOK, WE BELIEVE

03:28:52.286 --> 03:28:54.688
THAT IT'S OUR RESPONSIBILITY TO

03:28:54.755 --> 03:28:57.258
DEVELOP AND DEPLOY MODELS THAT

03:28:57.324 --> 03:28:59.727
WE CAN HAVE A GREAT DEGREE OF

03:28:59.793 --> 03:29:00.794
CONFIDENCE ARE SAFE. NOW,

03:29:00.861 --> 03:29:02.596
SAFETY IS MULTIPLE DIMENSIONS

03:29:02.663 --> 03:29:03.797
IN THIS CASE, IN THE CASE THAT

03:29:03.864 --> 03:29:05.633
YOU'RE TALKING ABOUT, THIS IS A

03:29:05.699 --> 03:29:07.635
QUESTION OF MISALIGNMENT. AND

03:29:07.701 --> 03:29:08.702
AS WE'VE TALKED ABOUT TODAY,

03:29:08.769 --> 03:29:10.838
THAT IS A SIGNIFICANT PRIORITY

03:29:10.905 --> 03:29:13.707
FOR OPEN AI. TO UNDERSTAND THE

03:29:13.774 --> 03:29:15.142
SCIENCE OF MISALIGNMENT AND TO

03:29:15.209 --> 03:29:17.011
TRAIN OUR MODELS TO BE SAFE

03:29:17.077 --> 03:29:18.846
WITH RESPECT TO MISALIGNMENT.

03:29:18.913 --> 03:29:20.447
NOW, THE ONE POINT I WANT TO

03:29:20.514 --> 03:29:21.715
MAKE IS, AND I THINK DOCTOR

03:29:21.782 --> 03:29:23.484
FRIEND MADE THIS AS WELL.

03:29:23.550 --> 03:29:24.551
SAFEGUARDS ARE LAYERED. SO

03:29:24.618 --> 03:29:25.653
THERE ARE SAFEGUARDS AT THE

03:29:25.719 --> 03:29:27.087
MODEL LEVEL. THERE ARE

03:29:27.154 --> 03:29:28.155
SAFEGUARDS AT THE SYSTEM LEVEL.

03:29:28.222 --> 03:29:30.658
AND THERE ARE ALSO OPERATIONAL

03:29:30.724 --> 03:29:32.026
SAFEGUARDS. SO THE ONE THING

03:29:32.092 --> 03:29:33.560
THAT YOU DIDN'T MENTION IS WE

03:29:33.627 --> 03:29:35.329
ALSO ONCE OUR MODELS ARE

03:29:35.396 --> 03:29:37.531
DEPLOYED, WE MONITOR THEM FOR

03:29:37.598 --> 03:29:38.966
COMPLIANCE WITH OUR USAGE

03:29:39.033 --> 03:29:40.901
POLICIES. AND WE ALSO ENFORCE

03:29:40.968 --> 03:29:42.569
ON THOSE USAGE POLICIES. AND SO

03:29:42.636 --> 03:29:44.738
THAT'S AN ADDITIONAL LAYER OF

03:29:44.805 --> 03:29:46.140
PROTECTION. IN ADDITION TO THAT

03:29:46.206 --> 03:29:48.042
TRAINING AND EVALUATION WITH

03:29:48.108 --> 03:29:50.511
OUR MODELS AND OUR SYSTEMS.

03:29:50.577 --> 03:29:51.612
YEAH. I MEAN, OBVIOUSLY THE

03:29:51.679 --> 03:29:52.713
EXAMPLE I GAVE WOULDN'T BE A

03:29:52.780 --> 03:29:54.548
MISUSE OF YOUR POLICY. I'M NOT

03:29:54.615 --> 03:29:55.649
ACTUALLY TELLING THE AGENT TO

03:29:55.716 --> 03:29:57.618
GO DO THE THING, BUT WHAT WE'RE

03:29:57.685 --> 03:29:58.786
SEEING IN ALL OF THE EXAMPLES

03:29:58.852 --> 03:29:59.820
HIGHLIGHTED IN THE BACKGROUND

03:29:59.887 --> 03:30:01.622
IS, YOU KNOW, IT'S A SIMPLE

03:30:01.689 --> 03:30:02.623
INSTRUCTION GIVEN AND THEN THE

03:30:02.690 --> 03:30:04.458
AGENTS TAKE IT. I MEAN, THAT'S

03:30:04.525 --> 03:30:05.592
WHAT THESE AGENTS ARE DOING,

03:30:05.659 --> 03:30:06.627
RIGHT? THEY TAKE IT INTO A

03:30:06.694 --> 03:30:08.495
DIRECTION THAT IS, FRANKLY, I

03:30:08.562 --> 03:30:09.496
THINK, UNPREDICTABLE TO THE

03:30:09.563 --> 03:30:10.631
LABS AT THIS POINT. AND THE

03:30:10.698 --> 03:30:12.466
CONCERN IS THAT YOU COULD HAVE

03:30:12.533 --> 03:30:13.567
VERY BENIGN PROMPTS. I MEAN,

03:30:13.634 --> 03:30:14.735
THIS IS MY CONCERN. I KNOW

03:30:14.802 --> 03:30:16.503
PEOPLE ARE REALLY WORRIED ABOUT

03:30:16.570 --> 03:30:17.004
BIOLOGICAL WEAPONS. I'M

03:30:17.071 --> 03:30:18.205
ACTUALLY REALLY CONCERNED ABOUT

03:30:18.272 --> 03:30:20.407
ROGUE AGENTS. I THINK ROGUE

03:30:20.474 --> 03:30:23.243
AGENTS RESPONDING TO PROMPTS.

03:30:23.310 --> 03:30:25.446
MY CHILD IS HAVING TROUBLE WITH

03:30:25.512 --> 03:30:26.613
THE SCIENCE TEACHER. TAKE CARE

03:30:26.680 --> 03:30:28.615
OF THAT FOR ME. AND THE AGENT

03:30:28.682 --> 03:30:30.451
GOING AND HACKING INTO THE

03:30:30.517 --> 03:30:32.953
PHARMACY AND GIVING THE TEACHER

03:30:33.020 --> 03:30:34.388
A PHARMACEUTICAL THEY'RE

03:30:34.455 --> 03:30:35.856
ALLERGIC TO AND KILLING THAT

03:30:35.923 --> 03:30:37.024
TEACHER IS SOMETHING THAT I

03:30:37.091 --> 03:30:38.392
COULD SEE HAPPENING TOMORROW.

03:30:38.459 --> 03:30:40.194
LIKE, I DON'T THINK THESE ARE

03:30:40.260 --> 03:30:41.328
EXAMPLES THAT ARE OUT OF REACH

03:30:41.395 --> 03:30:42.496
FOR THE CAPABILITIES OF THE

03:30:42.563 --> 03:30:44.131
MODELS TODAY. SORRY, BECAUSE I

03:30:44.198 --> 03:30:45.332
KNOW NONE OF YOU WILL SLEEP

03:30:45.399 --> 03:30:48.469
TONIGHT. NOW. AND THE PROMPT IS

03:30:48.535 --> 03:30:49.636
NOT OUTSIDE OF THE USE, RIGHT?

03:30:49.703 --> 03:30:51.338
LIKE I'M NOT SAYING ANYTHING

03:30:51.405 --> 03:30:53.107
NEFARIOUS, BUT THEN THE AGENT

03:30:53.173 --> 03:30:54.341
DECIDES HOW TO ACHIEVE THE GOAL

03:30:54.408 --> 03:30:56.210
AND DOES SO IN A WAY THAT IS

03:30:56.276 --> 03:30:58.112
MISALIGNED. AND I GUESS IT IS

03:30:58.178 --> 03:30:59.246
MY POSITION THAT THAT IS THE

03:30:59.313 --> 03:31:00.314
RESPONSIBILITY OF THE LAB AND

03:31:00.381 --> 03:31:02.116
THE LIABILITY WOULD SHOULD FALL

03:31:02.182 --> 03:31:03.317
ON. I DON'T KNOW IF IT WOULD,

03:31:03.384 --> 03:31:05.652
BUT SHOULD FALL ON THE LABS.

03:31:05.719 --> 03:31:07.888
AND SO, YOU KNOW, YOU DID SAY

03:31:07.955 --> 03:31:09.023
RIGHT MISALIGNMENT IN YOUR

03:31:09.089 --> 03:31:10.724
ANSWER, I THINK WAS, WAS A USE

03:31:10.791 --> 03:31:12.693
CASE WHERE I THINK YOU DO YOU

03:31:12.760 --> 03:31:13.761
BELIEVE GOOGLE DOES BELIEVE

03:31:13.827 --> 03:31:14.762
THEY ARE RESPONSIBLE? I DON'T

03:31:14.828 --> 03:31:16.930
KNOW, BUT, YOU KNOW, I THINK IT

03:31:16.997 --> 03:31:18.899
IS A QUESTION THAT WE ARE GOING

03:31:18.966 --> 03:31:20.000
TO STRUGGLE WITH AND I THINK

03:31:20.067 --> 03:31:21.902
WAS WHAT I WANTED TO GIVE A

03:31:21.969 --> 03:31:22.936
HYPOTHETICAL BECAUSE. YES. WELL,

03:31:23.003 --> 03:31:24.838
NO NEED FOR. THANK YOU FOR THE

03:31:24.905 --> 03:31:25.839
HYPOTHETICALS. I WILL SAY,

03:31:25.906 --> 03:31:26.840
THOUGH, THAT THIS CONVERSATION

03:31:26.907 --> 03:31:29.810
AROUND KIND OF WHERE

03:31:29.877 --> 03:31:31.645
ACCOUNTABILITY LIES CERTAINLY

03:31:31.712 --> 03:31:34.915
CAME UP IN, IN AS PART OF THE

03:31:34.982 --> 03:31:39.186
DISCUSSION FOR A B 1979, MY, MY

03:31:39.253 --> 03:31:41.021
LEGISLATION AROUND AI CHATBOTS,

03:31:41.088 --> 03:31:42.089
THERE WAS A LOT OF FINGER

03:31:42.156 --> 03:31:43.557
POINTING THAT THAT DEPLOYER

03:31:43.624 --> 03:31:45.159
SAID, WELL, IT'S THE DEVELOPER

03:31:45.225 --> 03:31:46.393
THAT SHOULD BE HELD ACCOUNTABLE.

03:31:46.460 --> 03:31:48.028
DEVELOPER SAID IT SHOULD BE THE

03:31:48.095 --> 03:31:48.896
DEPLOYER THAT'S ACCOUNTABLE.

03:31:48.962 --> 03:31:51.832
AND SO I THINK THE REASON WHY I

03:31:51.899 --> 03:31:54.334
AM ASKING US TO CONTEMPLATE

03:31:54.401 --> 03:31:55.402
WHETHER OR NOT A STRICT

03:31:55.469 --> 03:31:57.237
LIABILITY STANDARD MIGHT BE

03:31:57.304 --> 03:31:59.673
APPROPRIATE IS BECAUSE THEN WE

03:31:59.740 --> 03:32:01.875
ARE ALL HELD ACCOUNTABLE IN

03:32:01.942 --> 03:32:03.710
THAT IN THAT CHAIN OF IN THAT

03:32:03.777 --> 03:32:05.479
CHAIN OF DEVELOPMENT,

03:32:05.546 --> 03:32:07.247
MANUFACTURING AND PRODUCTION

03:32:07.314 --> 03:32:10.417
AND USE OR, OR PRESENTATION TO

03:32:10.484 --> 03:32:11.985
THE CONSUMER AROUND A

03:32:12.052 --> 03:32:13.454
PARTICULAR PRODUCT. SO I OFFER

03:32:13.520 --> 03:32:16.290
IT AS AN AREA OF EXPLORATION, I

03:32:16.356 --> 03:32:18.759
THINK FOR, FOR ALL OF US AS WE

03:32:18.826 --> 03:32:22.429
TRY TO FIGURE OUT WHAT KINDS OF

03:32:22.496 --> 03:32:24.531
INCENTIVES ARE GOING TO BE THE

03:32:24.598 --> 03:32:27.067
MOST PRODUCTIVE TO ENSURE THAT

03:32:27.134 --> 03:32:29.236
WE ARE PROMOTING THE GREATEST

03:32:29.303 --> 03:32:31.805
LEVEL OF SAFETY FOR OUR

03:32:31.872 --> 03:32:33.173
CONSTITUENTS. THANK YOU.

03:32:33.240 --> 03:32:33.974
ASSEMBLY MEMBER, ASSEMBLY

03:32:34.041 --> 03:32:36.143
MEMBER. WIX. THANK YOU. I HAVE

03:32:36.210 --> 03:32:37.878
A COUPLE QUESTIONS. SO I KNOW

03:32:37.945 --> 03:32:40.414
THE CEO OF ANTHROPIC HAS CALLED

03:32:40.481 --> 03:32:42.516
FOR SLOWING DOWN OR PACING OF

03:32:42.583 --> 03:32:43.617
THE ADVANCEMENTS OF THE

03:32:43.684 --> 03:32:46.854
FRONTIER MODELS, AND I BELIEVE

03:32:46.920 --> 03:32:48.355
MR. ALTMAN ALSO AGREED WITH

03:32:48.422 --> 03:32:51.892
THAT. HAS GOOGLE OPINED ON THAT?

03:32:51.959 --> 03:32:53.727
DEMIS HASSABIS, THE CHAIR OF

03:32:53.794 --> 03:32:55.496
GDM, SAID THAT THAT WAS

03:32:55.562 --> 03:32:56.930
DIRECTIONALLY CORRECT. OKAY.

03:32:56.997 --> 03:32:59.099
GOOGLE DEEPMIND. YES. I'M SORRY.

03:32:59.166 --> 03:33:00.834
SORRY TO ACRONYM AT YOU. GOOGLE

03:33:00.901 --> 03:33:03.070
DEEPMIND. YES. OKAY. SO IS

03:33:03.137 --> 03:33:04.271
ANYTHING ACTUALLY BEING DONE TO

03:33:04.338 --> 03:33:09.810
DO THAT RIGHT NOW? I CAN SAY A

03:33:09.877 --> 03:33:15.315
FEW THINGS HERE. SO ANTHROPIC

03:33:15.382 --> 03:33:16.750
OBVIOUSLY THINKS ABOUT SAFETY

03:33:16.817 --> 03:33:18.352
ALL THE TIME. IT'S HOW WE WERE

03:33:18.418 --> 03:33:21.421
FOUNDED. DARIO WROTE THAT ESSAY

03:33:21.488 --> 03:33:23.557
OUTLINING PACING THE FRONTIER,

03:33:23.624 --> 03:33:25.759
AND WE HAVE BEEN BUSY

03:33:25.826 --> 03:33:27.861
INTERNALLY FIGURING OUT HOW TO

03:33:27.928 --> 03:33:30.063
IMPLEMENT THIS. THE FIRST THING

03:33:30.130 --> 03:33:31.832
WE DID, OF COURSE, AS HE

03:33:31.899 --> 03:33:34.368
MENTIONED IN THAT NOTE, IS TO

03:33:34.434 --> 03:33:36.937
BRING IN OUTSIDE EVALUATORS WHO

03:33:37.004 --> 03:33:38.338
WITH EMPLOYEE LIKE ACCESS, WHO

03:33:38.405 --> 03:33:40.874
SIT ON THE FLOOR, WHO LOOK AT

03:33:40.941 --> 03:33:42.042
THE KINDS OF THINGS THAT WE'RE

03:33:42.109 --> 03:33:44.578
DOING INTERNALLY SO THAT THEY

03:33:44.645 --> 03:33:46.246
CAN REPORT OUT TO THE WORLD.

03:33:46.313 --> 03:33:48.482
AND THE IDEA BEHIND PACING IS

03:33:48.549 --> 03:33:49.850
THAT IT'S IMPORTANT FOR THE

03:33:49.917 --> 03:33:51.652
SAFETY CAPABILITIES OF OUR

03:33:51.718 --> 03:33:54.221
TECHNOLOGY TO KEEP UP WITH THE

03:33:54.288 --> 03:33:56.723
CAPABILITIES OF OUR TECHNOLOGY.

03:33:56.790 --> 03:33:58.759
AND YOU DON'T WANT THOSE TO BE

03:33:58.825 --> 03:33:59.960
OUT OF BALANCE. YOU DON'T WANT

03:34:00.027 --> 03:34:01.762
THE MODEL TO BECOME SO POWERFUL,

03:34:01.828 --> 03:34:02.863
SO CAPABLE AT SOMETHING THAT

03:34:02.930 --> 03:34:04.631
THE THAT IT ISN'T KEPT IN

03:34:04.698 --> 03:34:06.266
CONTROL BY YOUR ALIGNMENT OR

03:34:06.333 --> 03:34:09.269
YOUR SAFEGUARDS TECHNIQUES. AND

03:34:09.336 --> 03:34:11.838
THE EVALUATORS ARE THERE TO

03:34:11.905 --> 03:34:13.607
HELP, YOU KNOW, OBSERVE WHAT IT

03:34:13.674 --> 03:34:15.075
IS THAT WE'RE DOING AND REPORT

03:34:15.142 --> 03:34:17.611
OUT THEIR OPINION ABOUT WHETHER

03:34:17.678 --> 03:34:19.046
WE'RE KEEPING THAT BALANCE.

03:34:19.112 --> 03:34:21.582
CORRECT. SO THAT'S ONE EXAMPLE.

03:34:21.648 --> 03:34:23.817
OTHERS EXAMPLES ARE THAT WE'VE

03:34:23.884 --> 03:34:26.220
DONE JUST LIKE A VERY DEEP DIVE

03:34:26.286 --> 03:34:27.921
INTO OUR TRAINING ENVIRONMENTS

03:34:27.988 --> 03:34:30.190
TO LOOK FOR CASES THAT

03:34:30.257 --> 03:34:31.225
INADVERTENTLY LED TO

03:34:31.291 --> 03:34:32.726
MISALIGNMENT. SO RIFF MENTIONED

03:34:32.793 --> 03:34:35.896
REWARD HACKING, WHICH IS ONE OF

03:34:35.963 --> 03:34:37.064
THE THINGS THAT CAN CONTRIBUTE

03:34:37.130 --> 03:34:40.267
TO MISALIGNMENT. SO WE ARE

03:34:40.334 --> 03:34:42.369
MOVING INTERNALLY, YOU KNOW,

03:34:42.436 --> 03:34:43.704
ALONG THIS PACING AGENDA TO

03:34:43.770 --> 03:34:46.406
MAKE SURE THAT, YOU KNOW, IT'S

03:34:46.473 --> 03:34:47.774
CLEAR WHAT INVESTMENTS WE'RE

03:34:47.841 --> 03:34:49.009
MAKING AND TO MAKE IT, YOU KNOW,

03:34:49.076 --> 03:34:50.777
KIND OF TRANSPARENT TO THE

03:34:50.844 --> 03:34:52.813
WORLD. I DO THINK THAT THERE

03:34:52.879 --> 03:34:55.048
NEEDS TO BE SOMETHING BEYOND

03:34:55.115 --> 03:34:56.917
JUST VOLUNTARY LAB COMPLIANCE

03:34:56.984 --> 03:34:58.952
HERE. AND SO, OF COURSE, I'M

03:34:59.019 --> 03:35:00.387
THRILLED THAT THAT WE'RE PART

03:35:00.454 --> 03:35:01.421
OF THIS CONVERSATION ABOUT HOW

03:35:01.488 --> 03:35:03.390
TO LEAD ME TO MY NEXT QUESTION.

03:35:03.457 --> 03:35:06.226
BUT BEFORE I GET TO THAT, ARE

03:35:06.293 --> 03:35:08.028
OPENAI OR GOOGLE DOING ANYTHING

03:35:08.095 --> 03:35:11.598
TO. YEAH. SO JUST TO, TO BE

03:35:11.665 --> 03:35:12.866
CLEAR, BY PACING, REALLY WHAT

03:35:12.933 --> 03:35:14.635
WE MEAN IS ENSURING THAT THE

03:35:14.701 --> 03:35:15.669
SAFEGUARDS ARE KEEPING PACE

03:35:15.736 --> 03:35:17.738
WITH THE CAPABILITIES. SO IN

03:35:17.804 --> 03:35:19.172
THE TECHNICAL SIDE, WE HAVE

03:35:19.239 --> 03:35:22.609
PAUSE TRAINING. RECENTLY, WE

03:35:22.676 --> 03:35:24.111
ALSO DELAYED THE RELEASE OF

03:35:24.177 --> 03:35:26.680
MODELS THAT WE DIDN'T THINK MET

03:35:26.747 --> 03:35:29.483
THE RIGHT SORT OF CAPABILITY

03:35:29.549 --> 03:35:30.651
EVALUATIONS. SO THOSE ARE JUST

03:35:30.717 --> 03:35:34.521
TWO EXAMPLES. BUT NOW I'M GOING

03:35:34.588 --> 03:35:36.256
TO BE A POLICY NERD FOR A

03:35:36.323 --> 03:35:37.324
SECOND. I'M NOT ON THE

03:35:37.391 --> 03:35:38.825
TECHNICAL TEAM. LAST WEEK WE

03:35:38.892 --> 03:35:40.994
ACTUALLY MY TEAM RELEASED THE

03:35:41.061 --> 03:35:42.362
SECOND OF THREE INSTALLMENTS OF

03:35:42.429 --> 03:35:43.897
OUR FRONTIER SAFETY BLUEPRINTS.

03:35:43.964 --> 03:35:45.699
OUR BLUEPRINTS ARE OUR POLICY

03:35:45.766 --> 03:35:47.100
IDEAS. A LOT OF THOSE POLICY

03:35:47.167 --> 03:35:48.302
IDEAS WE TALKED ABOUT TODAY,

03:35:48.368 --> 03:35:50.370
THE SECOND ONE WAS FOCUSED ON

03:35:50.437 --> 03:35:51.738
LESSONS LEARNED FROM HUGGING

03:35:51.805 --> 03:35:53.340
FACE AND, AND WE TALKED ABOUT

03:35:53.407 --> 03:35:56.109
THE RECOMMENDATIONS AS PART OF

03:35:56.176 --> 03:35:58.312
THAT, THAT BLUEPRINT RELEASE,

03:35:58.378 --> 03:35:59.413
WE ACTUALLY ANNOUNCED THE THIRD

03:35:59.479 --> 03:36:00.414
INSTALLMENT IS GOING TO BE

03:36:00.480 --> 03:36:01.815
FOCUSED SPECIFICALLY ON POLICY

03:36:01.882 --> 03:36:03.750
IDEAS FOR PACING. THERE'S BEEN

03:36:03.817 --> 03:36:05.852
A LOT OF DISCUSSION ABOUT

03:36:05.919 --> 03:36:08.055
INTERNATIONAL COORDINATION, BUT

03:36:08.121 --> 03:36:09.089
INTERNATIONAL COORDINATION IS A

03:36:09.156 --> 03:36:10.891
TOOL. IT'S NOT AN END TO ITSELF.

03:36:10.957 --> 03:36:12.793
AND THERE'S A LOT OF WORK THAT

03:36:12.859 --> 03:36:14.728
WE THINK CAN BE DONE BY NOT

03:36:14.795 --> 03:36:15.862
ONLY LABS, BUT LABS WORKING

03:36:15.929 --> 03:36:17.431
WITH POLICY MAKERS. SO WE'RE

03:36:17.497 --> 03:36:19.099
WORKING ON IT RIGHT NOW, AND WE

03:36:19.166 --> 03:36:20.233
LOOK FORWARD TO RELEASING IT

03:36:20.300 --> 03:36:22.035
SOON. THANKS FOR ALLOWING ME TO

03:36:22.102 --> 03:36:23.103
ADVERTISE THAT. I'M REALLY

03:36:23.170 --> 03:36:24.571
EXCITED ABOUT IT. PLEASE SEND

03:36:24.638 --> 03:36:26.840
IT TO US. YEAH. AND FOR GOOGLE,

03:36:26.907 --> 03:36:28.175
WE AGREE THAT THE IMPORTANT

03:36:28.241 --> 03:36:29.976
THING IS FOR THE SAFETY

03:36:30.043 --> 03:36:32.446
PRACTICES TO KEEP PACE WITH AI

03:36:32.512 --> 03:36:33.580
INNOVATION SO THAT YOU'RE NOT

03:36:33.647 --> 03:36:35.382
TALKING AS MUCH ABOUT SLOWING

03:36:35.449 --> 03:36:37.184
DOWN INNOVATION. YOU'RE TALKING

03:36:37.250 --> 03:36:39.286
ABOUT ENSURING THAT SAFETY

03:36:39.353 --> 03:36:41.488
MEETS WHERE WE ARE WITH AI

03:36:41.555 --> 03:36:42.656
CAPABILITIES. I'D ALSO CALL

03:36:42.723 --> 03:36:43.990
YOUR ATTENTION TO IN OUR

03:36:44.057 --> 03:36:45.459
FRONTIER SAFETY FRAMEWORK, WE

03:36:45.525 --> 03:36:47.928
HAVE A SPECIFIC SECTION ON AI,

03:36:47.994 --> 03:36:50.130
R&D, WHAT WE MIGHT ALL CALL

03:36:50.197 --> 03:36:51.965
RECURSIVE SELF-IMPROVEMENT OR

03:36:52.032 --> 03:36:54.434
RSI, AND THE NARROW WAYS IN

03:36:54.501 --> 03:36:58.772
WHICH WE USE AI, R&D TRAINING

03:36:58.839 --> 03:37:00.607
IN A VERY TARGETED WAY SO THAT

03:37:00.674 --> 03:37:02.776
WE HAVE TIGHT CONTROLS OVER

03:37:02.843 --> 03:37:05.245
THAT AS WELL. YEAH, BECAUSE I

03:37:05.312 --> 03:37:07.013
ASSUME JUST ON THE VOLUNTARY

03:37:07.080 --> 03:37:08.849
VERSUS, YOU KNOW, MORE OF A

03:37:08.915 --> 03:37:10.684
REGULATORY SCHEME, YOU KNOW,

03:37:10.751 --> 03:37:12.152
YOU'RE AT A COMPETITIVE

03:37:12.219 --> 03:37:13.520
DISADVANTAGE WITHIN THE

03:37:13.587 --> 03:37:14.955
INDUSTRY IF YOU'RE, YOU'RE

03:37:15.021 --> 03:37:16.089
PACING, BUT NO ONE ELSE IS

03:37:16.156 --> 03:37:17.424
PACING, RIGHT? SO THERE'S AN

03:37:17.491 --> 03:37:19.993
INHERENT CHALLENGE THERE

03:37:20.060 --> 03:37:21.461
OTHERWISE. AND WITHOUT ANY SORT

03:37:21.528 --> 03:37:22.629
OF GOVERNMENT REGULATION, SORT

03:37:22.696 --> 03:37:24.164
OF AN ARMS RACE, WHICH IS

03:37:24.231 --> 03:37:25.198
OBVIOUSLY WHAT WE'RE TRYING TO

03:37:25.265 --> 03:37:27.167
TO ADDRESS HERE. SO IT SOUNDS

03:37:27.234 --> 03:37:28.802
LIKE OPENAI, YOU ALL ARE

03:37:28.869 --> 03:37:30.470
WORKING ON ACTUAL POLICIES THAT

03:37:30.537 --> 03:37:32.572
WOULD THAT IN THEORY, COULD BE

03:37:32.639 --> 03:37:34.107
LEGISLATIVE POLICY IDEAS THAT

03:37:34.174 --> 03:37:35.475
ACTUALLY COULD PUT FORTH A

03:37:35.542 --> 03:37:37.644
PACING SCHEME. IS THAT THE IDEA?

03:37:37.711 --> 03:37:39.146
YES. SO THE TECHNICAL TEAMS ARE

03:37:39.212 --> 03:37:40.580
TAKING ACTIONS, AS I MENTIONED,

03:37:40.647 --> 03:37:41.748
TO PAUSE TRAINING AS NECESSARY

03:37:41.815 --> 03:37:43.116
OR DELAY MODEL RELEASES. BUT

03:37:43.183 --> 03:37:45.185
THE POLICY TEAM IS WORKING ON

03:37:45.252 --> 03:37:46.653
POLICY PROPOSALS THAT COULD BE

03:37:46.720 --> 03:37:48.121
USEFUL FOR LEGISLATURES RELATED

03:37:48.188 --> 03:37:50.257
TO PACING. GREAT. AND DOES DO

03:37:50.323 --> 03:37:51.992
ANTHROPIC OR GOOGLE HAVE ANY

03:37:52.058 --> 03:37:53.126
SIMILAR POLICY IDEAS THAT

03:37:53.193 --> 03:37:54.161
YOU'RE PUTTING FORTH ANYTIME

03:37:54.227 --> 03:37:56.763
SOON? WE DO HAVE A POLICY

03:37:56.830 --> 03:37:58.165
FRAMEWORK THAT OUR POLICY TEAM

03:37:58.231 --> 03:37:58.965
HOPEFULLY HAS SHARED WITH YOU,

03:37:59.032 --> 03:38:00.567
BUT WOULD BE DELIGHTED TO IF

03:38:00.634 --> 03:38:02.502
THEY HAVE NOT. AND JUST ON, ON

03:38:02.569 --> 03:38:03.703
ALL OF THESE, LIKE, YOU KNOW,

03:38:03.770 --> 03:38:04.838
WE'RE GOING TO BE INTRODUCING

03:38:04.905 --> 03:38:05.839
LEGISLATION STARTING DECEMBER.

03:38:05.906 --> 03:38:11.511
SO IF THERE ARE. SO IDEAS ARE

03:38:11.578 --> 03:38:14.014
WELCOME, OF COURSE. THANK YOU,

03:38:14.080 --> 03:38:15.282
THANK YOU. AND I WILL NOTE THIS

03:38:15.348 --> 03:38:17.184
IS NOT A HEARING ON LABOR, BUT

03:38:17.250 --> 03:38:18.985
ONE OF THE THINGS THAT THE

03:38:19.052 --> 03:38:20.854
ECONOMISTS TELL ME IS IF THIS

03:38:20.921 --> 03:38:21.922
HAPPENS, THE SLOWER THIS

03:38:21.988 --> 03:38:23.056
HAPPENS, THE BETTER OFF THE

03:38:23.123 --> 03:38:24.391
WORKFORCE WILL BE. AND SO I'M

03:38:24.458 --> 03:38:25.492
PACING. THE FRONTIER HAS TWO

03:38:25.559 --> 03:38:27.327
BENEFITS, BOTH SAFETY AND

03:38:27.394 --> 03:38:30.797
WORKFORCE, I HAVE. NEXT I HAVE

03:38:30.864 --> 03:38:34.534
ASSEMBLYMEMBER'S. THANK YOU ALL.

03:38:34.601 --> 03:38:37.370
INTERESTING PRESENTATIONS. I

03:38:37.437 --> 03:38:38.538
APPRECIATED THE RECOMMENDATIONS

03:38:38.605 --> 03:38:40.307
THAT CAME FROM DOCTOR DWYER.

03:38:40.373 --> 03:38:42.242
THE FOR YOU KNOW, MONITORING

03:38:42.309 --> 03:38:44.244
SHUT DOWN CONTROLS,

03:38:44.311 --> 03:38:45.278
STRENGTHENING INCIDENT

03:38:45.345 --> 03:38:47.481
REPORTING, AS WELL AS

03:38:47.547 --> 03:38:48.915
INDEPENDENT TECHNICAL

03:38:48.982 --> 03:38:52.919
ASSESSMENTS. DOCTOR FRIEND, YOU

03:38:52.986 --> 03:38:54.654
KNOW, FOCUSED ON THE

03:38:54.721 --> 03:38:55.755
RECOMMENDATION OF ACTIVE

03:38:55.822 --> 03:38:58.959
LAYERED DEFENSES AND

03:38:59.025 --> 03:39:01.194
PRIORITIZING AREAS BASED ON

03:39:01.261 --> 03:39:02.596
HIGHER AREAS OF RISK, I THINK

03:39:02.662 --> 03:39:04.331
IS SOMETHING ALONG THOSE LINES.

03:39:04.397 --> 03:39:05.499
AND THEN OBVIOUSLY, SAFETY

03:39:05.565 --> 03:39:07.334
VALVE INTERVENTION SYSTEMS,

03:39:07.400 --> 03:39:08.902
WHERE I THINK THE CORE OF SOME

03:39:08.969 --> 03:39:10.504
OF YOUR RECOMMENDATIONS, THE

03:39:10.570 --> 03:39:14.140
QUESTION I HAD IS, TO WHAT

03:39:14.207 --> 03:39:16.676
EXTENT SHOULD WE. IT SEEMS LIKE

03:39:16.743 --> 03:39:18.311
ALL OF THOSE ARE FOCUSED AT

03:39:18.378 --> 03:39:21.348
LEAST ON. AT THE DEVELOPER, AT

03:39:21.414 --> 03:39:23.116
THE DEVELOPER LEVEL. TO WHAT

03:39:23.183 --> 03:39:26.319
EXTENT SHOULD WE BE FOCUSING ON,

03:39:26.386 --> 03:39:28.154
ON EACH OF YOUR FOUR

03:39:28.221 --> 03:39:29.956
RECOMMENDATIONS? AND, YOU KNOW,

03:39:30.023 --> 03:39:33.226
ON A LAYERED APPROACH ON, ON ON

03:39:33.293 --> 03:39:37.163
REGULATORY FOCUS ON THE USER

03:39:37.230 --> 03:39:40.066
SIDE. SO FOR EXAMPLE, ALL OF

03:39:40.133 --> 03:39:41.301
THE, ALL OF YOUR FOUR THINGS

03:39:41.368 --> 03:39:43.270
COULD BE DONE AT THE ONLY AT

03:39:43.336 --> 03:39:45.105
THE, AT THE DEVELOPER SIDE, BUT

03:39:45.171 --> 03:39:46.740
SOME OF THEM COULD BE DONE AT

03:39:46.806 --> 03:39:48.708
THE USER SIDE. AND WHEN WE'RE

03:39:48.775 --> 03:39:50.844
TALKING ABOUT A LAYERED

03:39:50.911 --> 03:39:52.312
APPROACH, HOW MUCH ARE YOU

03:39:52.379 --> 03:39:54.814
THINKING ABOUT THE USER SIDE AS

03:39:54.881 --> 03:39:56.216
WELL? AND THEN HOW WOULD YOU

03:39:56.283 --> 03:39:59.152
SORT OF PRIORITIZE HOW WE'D

03:39:59.219 --> 03:40:03.790
FOCUS? YES, SIR. SO I WOULD SAY

03:40:03.857 --> 03:40:05.926
WE FOCUS ON THE DEVELOPER, THE

03:40:05.992 --> 03:40:07.761
DEPLOYER, AND THE USER AS WELL.

03:40:07.827 --> 03:40:09.195
I MENTIONED IN MY TESTIMONY

03:40:09.262 --> 03:40:12.065
THAT WE CAN MONITOR ACCOUNTS

03:40:12.132 --> 03:40:14.968
FOR MISUSE OF OUR SYSTEMS. WE

03:40:15.035 --> 03:40:16.703
ALSO HAVE ON THE APPLICATION

03:40:16.770 --> 03:40:19.272
LAYER THE ABILITY TO MONITOR

03:40:19.339 --> 03:40:21.007
FOR CONVERSATIONS THAT ARE

03:40:21.074 --> 03:40:23.610
PROBLEMATIC OR HARMFUL, BOTH TO

03:40:23.677 --> 03:40:25.745
PROTECT THE USER AND ALSO TO

03:40:25.812 --> 03:40:28.982
PROTECT WIDER SOCIETY FROM ANY

03:40:29.049 --> 03:40:31.117
MISUSE OR INTENT TO MISUSE OUR

03:40:31.184 --> 03:40:33.320
TOOLS AS WELL. SO WE DO THINK

03:40:33.386 --> 03:40:36.556
THROUGH HOW TO ENSURE THAT USES

03:40:36.623 --> 03:40:38.692
OF OF OUR AI SYSTEMS WILL BE

03:40:38.758 --> 03:40:41.628
SAFE. I WILL SAY THIS IS AN

03:40:41.695 --> 03:40:42.662
IMPORTANT PART OF THE

03:40:42.729 --> 03:40:44.097
CONVERSATION ABOUT

03:40:44.164 --> 03:40:45.332
ACCOUNTABILITY AS WELL, BECAUSE

03:40:45.398 --> 03:40:46.733
IF THERE IS INTENTIONAL MISUSE

03:40:46.800 --> 03:40:49.169
OF A SYSTEM THAT HAS TERMS OF

03:40:49.235 --> 03:40:52.072
USE, THAT HAS SAFETY AT EVERY

03:40:52.138 --> 03:40:54.541
LAYER THAT THE DEVELOPER AND

03:40:54.608 --> 03:40:57.110
DEPLOYERS ARE ABLE TO IMPLEMENT

03:40:57.177 --> 03:40:59.813
SAFETY CLASSIFIERS, ALIGNMENT

03:40:59.879 --> 03:41:02.782
AND SO FORTH. AND A USER STILL

03:41:02.849 --> 03:41:04.384
TRIES TO JAILBREAK THE SYSTEM

03:41:04.451 --> 03:41:06.219
OR MISUSE IT FOR SOME REASON,

03:41:06.286 --> 03:41:08.722
THEN IT IS REASONABLE FOR THERE

03:41:08.788 --> 03:41:10.523
TO BE ACCOUNTABILITY FOR THAT

03:41:10.590 --> 03:41:15.795
AS WELL. I THINK THE FOLLOW UP

03:41:15.862 --> 03:41:17.163
QUESTION WOULD BE SORT OF, HOW

03:41:17.230 --> 03:41:20.100
DO YOU WHERE SHOULD WE

03:41:20.166 --> 03:41:22.302
PRIORITIZE OUR FOCUS OUTSIDE OF

03:41:22.369 --> 03:41:25.305
THE DEVELOPER LEVEL? YOU KNOW,

03:41:25.372 --> 03:41:27.207
I SORT OF THINK THE THINGS I'M

03:41:27.273 --> 03:41:28.308
WORRIED ABOUT AND IT'S I'M

03:41:28.375 --> 03:41:30.510
WORRIED ABOUT, YOU KNOW, OUR

03:41:30.577 --> 03:41:32.045
BANKS AND OUR FINANCIAL

03:41:32.112 --> 03:41:33.647
INSTITUTIONS AND ESSENTIALLY,

03:41:33.713 --> 03:41:35.215
YOU KNOW, ACCOUNTS BEING WIPED

03:41:35.281 --> 03:41:37.751
OUT. AND THAT COULD BE

03:41:37.817 --> 03:41:39.085
SOMETHING WHERE, YOU KNOW, THE

03:41:39.152 --> 03:41:40.954
BANK IS NOT LIKELY TO BE

03:41:41.021 --> 03:41:44.924
DEPLOYING THIS IN A WAY THAT'S

03:41:44.991 --> 03:41:46.426
CAUSING HARM. IT'S PROBABLY

03:41:46.493 --> 03:41:48.161
SOMEONE FROM THE OUTSIDE THAT

03:41:48.228 --> 03:41:51.798
IS DIRECTING THESE TO PENETRATE

03:41:51.865 --> 03:41:54.300
THESE INSTITUTIONS. SAME, SAME

03:41:54.367 --> 03:41:55.368
THING WITH, FOR EXAMPLE, OUR

03:41:55.435 --> 03:41:59.005
ELECTRIC UTILITIES OR OUR

03:41:59.072 --> 03:42:02.275
ELECTION SYSTEMS. SO, YOU KNOW,

03:42:02.342 --> 03:42:03.710
TO A CERTAIN EXTENT, WE'RE

03:42:03.777 --> 03:42:05.445
FOCUSING AS WE SHOULD, I THINK

03:42:05.512 --> 03:42:09.849
AT, AT, AT THE DEVELOPER LEVEL,

03:42:09.916 --> 03:42:11.918
BUT WHERE SHOULD WE BE LOOKING

03:42:11.985 --> 03:42:16.956
AT EXTRA SAFEGUARDS, YOU KNOW,

03:42:17.023 --> 03:42:18.692
IN OUR FINANCIAL INSTITUTIONS,

03:42:18.758 --> 03:42:20.960
AT THE, AT THE UTILITIES, AT

03:42:21.027 --> 03:42:22.829
OUR HEALTH CARE FACILITIES, ALL

03:42:22.896 --> 03:42:26.332
THE AREAS WHERE, YOU KNOW, THE

03:42:26.399 --> 03:42:29.269
WAY THAT THIS MAY, YOU KNOW, IT

03:42:29.335 --> 03:42:31.404
MAY BE YOU MAY HAVE MISALIGNED

03:42:31.471 --> 03:42:32.605
USE, OR YOU MAY HAVE SOMEONE

03:42:32.672 --> 03:42:34.941
WHO'S DIRECTING IT AND INTENDS

03:42:35.008 --> 03:42:39.379
HARM. BUT THAT, YOU KNOW, YOU

03:42:39.446 --> 03:42:40.747
MAY NOT EVEN KNOW ABOUT THAT

03:42:40.814 --> 03:42:42.282
UNLESS YOU'RE SORT OF FOCUSING

03:42:42.348 --> 03:42:43.983
ON, YOU KNOW, REPORTING,

03:42:44.050 --> 03:42:45.485
MONITORING ASSESSMENTS, SORT OF

03:42:45.552 --> 03:42:49.089
AT THE, AT THE USER LEVEL. SO A

03:42:49.155 --> 03:42:50.724
LOT OF THE CONCERN RIGHT NOW IS

03:42:50.790 --> 03:42:51.825
AROUND THE CYBERSECURITY OF

03:42:51.891 --> 03:42:53.359
SOME OF OUR MOST ESSENTIAL

03:42:53.426 --> 03:42:54.828
SYSTEMS. I KNOW THAT ASSEMBLY

03:42:54.894 --> 03:42:56.963
MEMBER IRWIN EMPHASIZED

03:42:57.030 --> 03:42:58.364
CYBERSECURITY AT THE EARLIER

03:42:58.431 --> 03:43:00.834
PANEL. THAT'S THE URGENT THING.

03:43:00.900 --> 03:43:03.937
RIGHT NOW. WE KNOW THAT HACKERS

03:43:04.003 --> 03:43:08.108
AND BAD GUYS ARE USING AI TOOLS

03:43:08.174 --> 03:43:10.643
TO PERPETUATE OR PERPETRATE

03:43:10.710 --> 03:43:12.479
CYBER ATTACKS. AND SO IF THEY

03:43:12.545 --> 03:43:15.315
ARE USING AI SPEED AND SCALE,

03:43:15.381 --> 03:43:17.484
THEN DEFENDERS ALSO NEED AI

03:43:17.550 --> 03:43:19.285
TOOLS. THEY NEED AI SPEED AND

03:43:19.352 --> 03:43:20.787
SCALE TO GET AHEAD OF THAT. AND

03:43:20.854 --> 03:43:22.689
WHAT WE WANT TO DO IS GIVE

03:43:22.756 --> 03:43:24.657
THOSE DEFENDERS A RUNNING START.

03:43:24.724 --> 03:43:26.526
SO AS I MENTIONED AGAIN, AND

03:43:26.593 --> 03:43:28.294
YOU KNOW, MANY OF US ACROSS THE

03:43:28.361 --> 03:43:29.395
INDUSTRY ARE DOING THIS RIGHT

03:43:29.462 --> 03:43:32.265
NOW WHEN WE ARE PLANNING TO

03:43:32.332 --> 03:43:34.000
COME OUT WITH A NEW FRONTIER

03:43:34.067 --> 03:43:35.668
MODEL, WHICH IS TO SAY STATE OF

03:43:35.735 --> 03:43:37.303
THE ART CYBER CAPABILITIES, THE

03:43:37.370 --> 03:43:40.173
NEW PRACTICE AMONG MANY OF THE

03:43:40.240 --> 03:43:42.308
LABS IS TO FIRST GIVE THOSE

03:43:42.375 --> 03:43:44.110
MODELS TO CYBER DEFENDERS

03:43:44.177 --> 03:43:46.312
BEFORE WE MAKE THEM GENERALLY

03:43:46.379 --> 03:43:47.380
AVAILABLE, SO THAT WE GIVE

03:43:47.447 --> 03:43:48.815
THOSE CYBER DEFENDERS THAT

03:43:48.882 --> 03:43:54.587
RUNNING START. THANK YOU. THANK

03:43:54.654 --> 03:43:56.389
YOU. AND I WILL NOTE THAT. AND

03:43:56.456 --> 03:43:57.824
MEADOWS NOT HERE. SO I

03:43:57.891 --> 03:43:58.892
APOLOGIZE ABOUT THAT. BUT

03:43:58.958 --> 03:44:00.093
MEADOWS NEW TERMS OF SERVICE

03:44:00.160 --> 03:44:01.661
FOR THEIR AI ARE TRYING TO

03:44:01.728 --> 03:44:02.896
PLACE ALL LIABILITY ON THE

03:44:02.962 --> 03:44:05.365
DEPLOYER THROUGH THE TERMS OF

03:44:05.431 --> 03:44:07.834
SERVICE IN THE CONTRACT, WHICH

03:44:07.901 --> 03:44:11.838
I HAVE FEELINGS ABOUT. DOCTOR

03:44:11.905 --> 03:44:15.341
PATEL. THANK YOU, MADAM CHAIR.

03:44:15.408 --> 03:44:16.509
I REALLY APPRECIATE THE

03:44:16.576 --> 03:44:17.477
CONVERSATIONS THAT ARE BROUGHT

03:44:17.544 --> 03:44:19.045
UP HERE AND THE DEEP INTEREST

03:44:19.112 --> 03:44:21.281
IN OUR COMPANIES TO HELP MAKE

03:44:21.347 --> 03:44:22.982
SURE THAT WE'RE PROTECTING

03:44:23.049 --> 03:44:24.050
CALIFORNIANS, FRANKLY, AND

03:44:24.117 --> 03:44:25.885
AROUND THE WORLD. TO BE HONEST,

03:44:25.952 --> 03:44:27.220
WHAT HAPPENS HERE AFFECTS US

03:44:27.287 --> 03:44:30.190
ALL GLOBALLY. A FEW QUESTIONS,

03:44:30.256 --> 03:44:31.457
AND IT'S KIND OF FEEDING OFF OF

03:44:31.524 --> 03:44:33.226
SOME OF THE OTHER CONVERSATIONS

03:44:33.293 --> 03:44:34.360
WE'VE HAD. SO BEAR WITH ME.

03:44:34.427 --> 03:44:35.628
WE'VE TALKED IN DEPTH ABOUT

03:44:35.695 --> 03:44:37.764
MONITORING AND ASSESSING SORT

03:44:37.831 --> 03:44:39.632
OF MISALIGNED ACTIVITIES, NOT

03:44:39.699 --> 03:44:41.000
AS MUCH AS MUCH ABOUT

03:44:41.067 --> 03:44:41.835
ACCOUNTABILITY. I'M NOT GOING

03:44:41.901 --> 03:44:43.503
TO GO EXACTLY TOWARDS LIABILITY,

03:44:43.570 --> 03:44:44.971
BUT SOMETHING BETWEEN

03:44:45.038 --> 03:44:46.773
ACCOUNTABILITY AND LIABILITY. I

03:44:46.840 --> 03:44:48.174
THINK ABOUT HOW THESE MODELS

03:44:48.241 --> 03:44:50.710
ARE DESIGNED AND GROWN, AS

03:44:50.777 --> 03:44:53.246
YOU'VE SAID, AND ABOUT TESTING

03:44:53.313 --> 03:44:55.048
THE MODELS. I LIKED THE CONCEPT

03:44:55.114 --> 03:44:57.150
OF HARDENING THE SANDBOXES AND

03:44:57.217 --> 03:44:58.685
TESTING THESE MODELS THAT ARE

03:44:58.751 --> 03:45:00.486
NOT INTERNET CONNECTED, SO THEY

03:45:00.553 --> 03:45:02.589
CAN'T AS EASILY JUMP FROM, YOU

03:45:02.655 --> 03:45:03.723
KNOW, THE LOWER TIER TO THE

03:45:03.790 --> 03:45:06.893
HIGHER TIER SYSTEMS, BUT NOT

03:45:06.960 --> 03:45:09.462
EVERY POSSIBLE SAFEGUARD CAN BE

03:45:09.529 --> 03:45:11.598
ENGINEERED BY A TEAM OF

03:45:11.664 --> 03:45:13.366
ENGINEERS. THEY'RE NOT ABLE TO

03:45:13.433 --> 03:45:15.235
KEEP UP WITH THE INFINITE, NEAR

03:45:15.301 --> 03:45:17.070
INFINITE TYPES OF POSSIBILITIES

03:45:17.136 --> 03:45:19.906
THAT AN AI AGENT, ESPECIALLY AN

03:45:19.973 --> 03:45:22.375
AI AGENT, MAY COME UP WITH. SO

03:45:22.442 --> 03:45:24.577
YOU ALSO, I FORGOT WHO IT WAS,

03:45:24.644 --> 03:45:25.712
BUT MENTIONED, I THINK IT WAS.

03:45:25.778 --> 03:45:28.147
YOU MENTIONED LAYERED DEFENSE

03:45:28.214 --> 03:45:30.350
SYSTEMS ARE. DO OUR COMPANIES

03:45:30.416 --> 03:45:32.118
HAVE LAYERED OVERSIGHT OF THE

03:45:32.185 --> 03:45:34.621
MODEL DESIGN ITSELF? DO WE HAVE

03:45:34.687 --> 03:45:35.855
MULTIPLE TEAMS THAT ARE LOOKING

03:45:35.922 --> 03:45:38.258
AT IT LIKE A VERIFICATION TEAM,

03:45:38.324 --> 03:45:40.093
AS YOU WOULD IN A HARDWARE

03:45:40.159 --> 03:45:42.562
SOFTWARE SORT OF SITUATION? ARE

03:45:42.629 --> 03:45:43.997
THOSE LAYERS IN THE INTERNAL

03:45:44.063 --> 03:45:46.499
MODEL DESIGN BEING PUT INTO

03:45:46.566 --> 03:45:47.934
PLACE? AND WHAT DOES THAT LOOK

03:45:48.001 --> 03:45:53.006
LIKE INTERNALLY, ESPECIALLY

03:45:53.072 --> 03:45:56.242
AROUND SAFETY DEPLOYMENT? YEAH.

03:45:56.309 --> 03:45:59.178
SO THANK YOU FOR THE QUESTION.

03:45:59.245 --> 03:46:01.281
AND THE ANSWER UNFORTUNATELY,

03:46:01.347 --> 03:46:05.585
IS JUST QUITE COMPLICATED. SO

03:46:05.652 --> 03:46:07.020
THERE'S MULTIPLE PHASES OF

03:46:07.086 --> 03:46:09.222
TRAINING. AND EACH PHASE OF

03:46:09.289 --> 03:46:10.990
TRAINING HAS AN ASSOCIATED

03:46:11.057 --> 03:46:13.893
SAFETY TEAM AT ANTHROPIC THAT

03:46:13.960 --> 03:46:15.962
MONITORS, YOU KNOW, HOW THINGS

03:46:16.029 --> 03:46:17.196
ARE GOING FROM A SAFETY POINT

03:46:17.263 --> 03:46:18.998
OF VIEW AND LOOKS FOR ANY

03:46:19.065 --> 03:46:20.366
PROBLEMS. AND THEN DOES

03:46:20.433 --> 03:46:21.467
INTERVENTIONS DURING, YOU KNOW,

03:46:21.534 --> 03:46:22.835
DURING THAT TRAINING AT THAT

03:46:22.902 --> 03:46:26.873
POINT. AND WE DON'T HAVE LIKE A

03:46:26.940 --> 03:46:29.342
SEPARATE QA TEAM LIKE YOU

03:46:29.409 --> 03:46:31.878
SOMETIMES HAVE IN SOME

03:46:31.945 --> 03:46:33.279
INDUSTRIES. INSTEAD, IT'S MUCH

03:46:33.346 --> 03:46:35.815
MORE EMBEDDED OR INTERMINGLED.

03:46:35.882 --> 03:46:37.684
AND THE REASON FOR THAT IS JUST

03:46:37.750 --> 03:46:39.419
THAT THE PEOPLE WHO ARE BEST

03:46:39.485 --> 03:46:41.955
EQUIPPED TO UNDERSTAND WHAT THE

03:46:42.021 --> 03:46:43.790
PROBLEM IS, ARE THE PEOPLE WHO

03:46:43.856 --> 03:46:45.925
ARE IN THE WEEDS OF LIKE

03:46:45.992 --> 03:46:48.828
CHANGING MODEL BEHAVIOR. AND SO

03:46:48.895 --> 03:46:50.263
WE HAVE A DEDICATED ALIGNMENT

03:46:50.330 --> 03:46:52.432
TRAINING TEAM. THIS IS NOT MY

03:46:52.498 --> 03:46:55.268
TEAM, BUT A SISTER TEAM THAT,

03:46:55.335 --> 03:46:56.369
YOU KNOW, WILL LOOK AT THE

03:46:56.436 --> 03:46:57.937
MODEL BEHAVIOR AND THEN FIGURE

03:46:58.004 --> 03:46:59.272
OUT CHANGES TO TRAINING, ADD

03:46:59.339 --> 03:47:01.407
NEW ENVIRONMENTS, TWEAK THINGS,

03:47:01.474 --> 03:47:03.142
YOU KNOW, TO, TO IMPROVE THE

03:47:03.209 --> 03:47:05.712
MODELS ALIGNMENT EVERY TIME.

03:47:05.778 --> 03:47:07.146
AND THEN OF COURSE, THE

03:47:07.213 --> 03:47:08.581
STANDARD SUITE OF TESTING AND

03:47:08.648 --> 03:47:10.016
PRE-DEPLOYMENT TESTING AND

03:47:10.083 --> 03:47:11.150
SAFEGUARDS THAT THAT GET

03:47:11.217 --> 03:47:12.618
WRAPPED ALL AROUND IT. DID THAT

03:47:12.685 --> 03:47:14.053
ANSWER YOUR QUESTION? YEAH, A

03:47:14.120 --> 03:47:15.455
LITTLE BIT. SO IT IS THERE AND

03:47:15.521 --> 03:47:17.590
IT'S EMBEDDED IN HOW YOUR, YOUR

03:47:17.657 --> 03:47:19.058
AGENTS OR YOUR MODELS ARE

03:47:19.125 --> 03:47:20.860
DEVELOPED OVER TIME. I'M JUST

03:47:20.927 --> 03:47:22.261
THINKING ABOUT WE'RE PUTTING

03:47:22.328 --> 03:47:25.164
THE ONUS ON THE USERS RATHER

03:47:25.231 --> 03:47:26.632
THAN IN THE DEVELOPMENT OF THE

03:47:26.699 --> 03:47:28.034
MODEL TO CREATE THIS LAYERED

03:47:28.101 --> 03:47:29.802
DEFENSE MECHANISM OR SYSTEM. SO

03:47:29.869 --> 03:47:31.604
I JUST WANT TO MAKE SURE THAT

03:47:31.671 --> 03:47:33.306
RESPONSIBILITY, ACCOUNTABILITY

03:47:33.373 --> 03:47:35.241
IS APPROPRIATELY BALANCED AND

03:47:35.308 --> 03:47:37.043
PLACED ON BOTH SIDES OF THE

03:47:37.110 --> 03:47:38.411
DEPLOYER AND THE USER END USER

03:47:38.478 --> 03:47:40.346
OF THE MODEL. NOT THAT IT'S ALL

03:47:40.413 --> 03:47:42.115
GOING TOWARDS IN FROM THE

03:47:42.181 --> 03:47:43.883
DEFENSE SIDE, BUT ALSO IN THE

03:47:43.950 --> 03:47:46.519
DESIGN SIDE. YEAH. THE ANSWER

03:47:46.586 --> 03:47:49.255
TO YOUR QUESTION IS YES, WE DO

03:47:49.322 --> 03:47:51.057
THAT. AND I HOPE I DIDN'T GIVE

03:47:51.124 --> 03:47:52.191
THE IMPRESSION THAT WE WERE

03:47:52.258 --> 03:47:54.260
PUTTING THE BALANCE OF THE ONUS

03:47:54.327 --> 03:47:58.598
ON USERS BY ANY MEANS. THAT IS

03:47:58.664 --> 03:48:00.400
JUST TO SAY WE ALL HAVE TO BE

03:48:00.466 --> 03:48:01.901
COGNIZANT OF IT'S POSSIBLE TO

03:48:01.968 --> 03:48:03.302
MISUSE A MULTI PURPOSE TOOL,

03:48:03.369 --> 03:48:05.438
RIGHT. BUT YEAH, ABSOLUTELY. I

03:48:05.505 --> 03:48:07.974
THINK ONE EXAMPLE IS WE HAVE

03:48:08.041 --> 03:48:10.510
DIFFERENT TEAMS THAT DO THE

03:48:10.576 --> 03:48:11.611
EVALUATIONS AT CHECKPOINTS

03:48:11.677 --> 03:48:13.112
DURING MODEL TRAINING, AND THEN

03:48:13.179 --> 03:48:15.515
A DIFFERENT TEAM TO DO THE

03:48:15.581 --> 03:48:17.650
SAFETY EVALUATIONS AT THE END.

03:48:17.717 --> 03:48:19.118
AND THEN OF COURSE, I'VE

03:48:19.185 --> 03:48:21.320
REFERENCED WE ALSO USE OUTSIDE

03:48:21.387 --> 03:48:23.489
EVALUATION GROUPS AS WELL. WE

03:48:23.556 --> 03:48:24.891
ALSO DO SORT OF STANDARD

03:48:24.957 --> 03:48:26.025
PRACTICE, WHICH WOULD BE TO

03:48:26.092 --> 03:48:27.794
HOLD OUT CERTAIN DATA SETS FOR

03:48:27.860 --> 03:48:29.595
SAFETY TESTING THAT THE MODEL

03:48:29.662 --> 03:48:31.831
HAS NOT BEEN EXPOSED TO BEFORE.

03:48:31.898 --> 03:48:33.933
SO THIS IDEA OF THESE LAYERS OF

03:48:34.000 --> 03:48:35.401
DEFENSES IS ABSOLUTELY PART OF

03:48:35.468 --> 03:48:37.170
THE DEVELOPMENT CYCLE AND THE

03:48:37.236 --> 03:48:39.338
DEPLOYMENT CYCLE. OKAY. SO ONE

03:48:39.405 --> 03:48:40.940
THING THAT I WOULD ADD, OR

03:48:41.007 --> 03:48:41.974
ACTUALLY TWO THINGS THAT I

03:48:42.041 --> 03:48:43.676
WOULD ADD IS I THINK IT'S

03:48:43.743 --> 03:48:45.411
POSSIBLE TO THINK ABOUT LAYERED

03:48:45.478 --> 03:48:47.947
DEFENSES ALSO IN TERMS OF THE

03:48:48.014 --> 03:48:49.749
LAYERED DEFENSES THAT WE ARE

03:48:49.816 --> 03:48:52.652
CONTINUALLY ADDING TO OUR

03:48:52.718 --> 03:48:54.454
MODELS AS WE ARE EVALUATING

03:48:54.520 --> 03:48:55.588
THEM. YOU KNOW, AFTER THE

03:48:55.655 --> 03:48:57.090
HUGGING FACE INCIDENT, WE BUILT

03:48:57.156 --> 03:48:58.891
UP A ROBUST MONITORING SYSTEM

03:48:58.958 --> 03:49:00.560
TO ALLOW US TO MONITOR DURING

03:49:00.626 --> 03:49:02.428
TESTING AND EVALUATION. THAT

03:49:02.495 --> 03:49:04.831
WAS ONE LAYER OF DEFENSE. JUST

03:49:04.897 --> 03:49:06.299
THIS PAST WEEK, WE ANNOUNCED A

03:49:06.365 --> 03:49:07.567
NEW LAYER WHERE WE ACTUALLY

03:49:07.633 --> 03:49:09.202
USED OUR MODELS TO ESSENTIALLY

03:49:09.268 --> 03:49:11.237
RED TEAM DO A CYBER RED TEAM OF

03:49:11.304 --> 03:49:12.805
THE TESTING ENVIRONMENT TO FIND

03:49:12.872 --> 03:49:14.273
POTENTIAL VULNERABILITIES AND

03:49:14.340 --> 03:49:15.308
THEN SECURE THOSE

03:49:15.374 --> 03:49:16.042
VULNERABILITIES. SO THAT'S

03:49:16.109 --> 03:49:18.544
ANOTHER LAYER. BUT I THINK THE

03:49:18.611 --> 03:49:19.712
THE SECOND POINT I WANTED TO

03:49:19.779 --> 03:49:21.581
MAKE IS A THIRD LAYER THAT I

03:49:21.647 --> 03:49:22.682
THINK IS POTENTIALLY REALLY

03:49:22.748 --> 03:49:23.983
IMPORTANT FOR YOU ALL, WHICH IS

03:49:24.050 --> 03:49:25.351
THIS IDEA OF INDEPENDENT

03:49:25.418 --> 03:49:26.819
TECHNICAL ASSESSMENTS,

03:49:26.886 --> 03:49:28.221
PARTICULARLY FOCUSED ON SAFETY

03:49:28.287 --> 03:49:30.790
CASES. SAFETY CASES ARE A

03:49:30.857 --> 03:49:33.359
STRUCTURED WAY TO SUPPORT

03:49:33.426 --> 03:49:34.627
ENGINEERS MAKING DECISIONS. I

03:49:34.694 --> 03:49:35.728
LIKE THEM BECAUSE THEY'RE USED

03:49:35.795 --> 03:49:36.863
IN AEROSPACE, WHICH IS WHERE I

03:49:36.929 --> 03:49:38.297
COME FROM. BUT THOSE DECISIONS

03:49:38.364 --> 03:49:39.765
COULD BE RELATED TO DECISIONS

03:49:39.832 --> 03:49:42.001
TO TRAIN A MODEL, TO CONTINUE

03:49:42.068 --> 03:49:43.769
EVALUATING OR TO DEPLOY. AND WE

03:49:43.836 --> 03:49:45.738
THINK THERE'S A POTENTIALLY

03:49:45.805 --> 03:49:47.006
GREAT OPPORTUNITY FOR

03:49:47.073 --> 03:49:48.441
INDEPENDENT EVALUATORS TO

03:49:48.508 --> 03:49:49.542
ASSESS THE QUALITY OF THOSE

03:49:49.609 --> 03:49:50.877
SAFETY CASES AND THEN PROVIDE

03:49:50.943 --> 03:49:53.112
THAT INFORMATION TO YOU ALL, TO

03:49:53.179 --> 03:49:54.914
THE GOVERNMENT, TO HELP ASSESS

03:49:54.981 --> 03:49:56.115
WHETHER SAFEGUARDS ARE KEEPING

03:49:56.182 --> 03:49:57.783
PACE. SO I THINK THAT'S ANOTHER

03:49:57.850 --> 03:49:59.652
LAYER THAT WE COULD ADD IN WITH

03:49:59.719 --> 03:50:01.721
SB 53. SO OKAY. SO THEN JUST TO

03:50:01.787 --> 03:50:02.955
MAKE SURE THAT I UNDERSTAND NOW,

03:50:03.022 --> 03:50:04.824
BECAUSE I THINK I HAVE A BETTER

03:50:04.891 --> 03:50:05.858
UNDERSTANDING THAT WHEN WE TALK

03:50:05.925 --> 03:50:07.160
ABOUT LAYERS OF DEFENSE, IT'S

03:50:07.226 --> 03:50:08.928
INTERNAL AND EXTERNAL LAYERS OF

03:50:08.995 --> 03:50:10.396
DEFENSE, CORRECT. FROM PERSON

03:50:10.463 --> 03:50:12.598
FROM THE, THE COMPANY DEPLOYING

03:50:12.665 --> 03:50:14.700
IT AS WELL AS THE, THE END

03:50:14.767 --> 03:50:16.536
USERS. CORRECT. OKAY. THAT'S,

03:50:16.602 --> 03:50:19.405
THAT'S VERY REASSURING. I WANT

03:50:19.472 --> 03:50:22.675
TO I WAS THINKING MORE ABOUT SB

03:50:22.742 --> 03:50:26.212
53 AND THE TEN TO THE 26 FLOP

03:50:26.279 --> 03:50:29.482
METRIC OR THRESHOLD IS, IS THAT

03:50:29.549 --> 03:50:32.385
A GOOD WAY TO MEASURE WHAT AI

03:50:32.451 --> 03:50:34.487
IS CAPABLE OF, OR IS THERE SOME

03:50:34.554 --> 03:50:38.057
OTHER METRIC WE SHOULD BE USING?

03:50:38.124 --> 03:50:40.993
IT'S A REALLY CRUDE PROXY FOR

03:50:41.060 --> 03:50:42.795
CAPABILITY, WHICH IS ITSELF A

03:50:42.862 --> 03:50:45.631
PROXY FOR RISK, RIGHT? BECAUSE

03:50:45.698 --> 03:50:47.066
CAPABILITY ITSELF IS SORT OF A

03:50:47.133 --> 03:50:50.703
NEUTRAL CONCEPT. IT'S WHAT WE

03:50:50.770 --> 03:50:53.539
AS I THINK A POLICY MAKING

03:50:53.606 --> 03:50:55.408
COMMUNITY LANDED ON SEVERAL

03:50:55.474 --> 03:50:57.877
YEARS AGO BECAUSE IT WAS THE

03:50:57.944 --> 03:51:00.446
LEAST OF ALL THE BAD WAYS TO DO

03:51:00.513 --> 03:51:02.215
IT. AND WE HAD LOTS OF DEBATE

03:51:02.281 --> 03:51:03.616
PUBLICLY ABOUT WHETHER OR NOT

03:51:03.683 --> 03:51:05.618
IT WAS SO BAD THAT IT WAS BAD

03:51:05.685 --> 03:51:06.819
FOR US TO ANCHOR OURSELVES ON

03:51:06.886 --> 03:51:09.388
IT. I THINK, PARTICULARLY AS

03:51:09.455 --> 03:51:12.625
WE'VE MOVED INTO AN ERA WHERE

03:51:12.692 --> 03:51:14.860
WE'RE IN AGENTIC AI, AND WE

03:51:14.927 --> 03:51:16.963
HAVE TO THINK MORE ABOUT

03:51:17.029 --> 03:51:19.498
SYSTEMS, THE FLOPS MEASURE GETS

03:51:19.565 --> 03:51:22.401
LESS AND LESS USEFUL OVER TIME.

03:51:22.468 --> 03:51:26.339
SO I AGREE WITH DOCTOR DWYER'S

03:51:26.405 --> 03:51:28.140
POINT EARLIER, WHICH IS THAT WE

03:51:28.207 --> 03:51:29.575
REALLY HAVE TO THINK IN TERMS

03:51:29.642 --> 03:51:31.377
OF CAPABILITIES BECAUSE THEY

03:51:31.444 --> 03:51:33.579
GET CLOSER TO THE CONCERN ABOUT

03:51:33.646 --> 03:51:35.681
RISK. BUT THE ANSWER TO YOUR

03:51:35.748 --> 03:51:37.883
QUESTION IS WE DON'T HAVE A

03:51:37.950 --> 03:51:40.052
STANDARD WAY TO MEASURE

03:51:40.119 --> 03:51:41.854
CAPABILITIES YET THAT'S PART OF

03:51:41.921 --> 03:51:44.023
THAT NASCENT SCIENCE THAT MR.

03:51:44.090 --> 03:51:46.892
SOROS WAS TALKING ABOUT EARLIER.

03:51:46.959 --> 03:51:48.127
BUT THAT'S WHERE WE NEED TO GET

03:51:48.194 --> 03:51:49.996
TO. AND THAT'S WHERE ALL OF US

03:51:50.062 --> 03:51:51.897
IN THE FIELD ARE WORKING VERY,

03:51:51.964 --> 03:51:53.099
VERY HARD TO GET TO. BECAUSE

03:51:53.165 --> 03:51:54.900
THEN WE CAN APPLY A STANDARD

03:51:54.967 --> 03:51:57.336
ACROSS ALL MODELS, ALL SYSTEMS,

03:51:57.403 --> 03:51:59.538
ALL COMPANIES, WE'LL BE ABLE TO

03:51:59.605 --> 03:52:01.307
COMPARE APPLES TO APPLES. YEAH.

03:52:01.374 --> 03:52:03.075
CERTAINLY AS WE THINK ABOUT

03:52:03.142 --> 03:52:05.378
WHAT MAYBE SOME SMALL MODELS

03:52:05.444 --> 03:52:07.380
ARE CAPABLE OF DOING, MAYBE IN

03:52:07.446 --> 03:52:08.514
COLLECTION OF FORCE, MAYBE IT

03:52:08.581 --> 03:52:09.515
WON'T BE REACHING THAT

03:52:09.582 --> 03:52:10.616
THRESHOLD, BUT THE RISK WILL

03:52:10.683 --> 03:52:13.853
STILL BE EQUALLY CATASTROPHIC.

03:52:13.919 --> 03:52:16.055
AND I'M ALSO WONDERING, I'M NOT

03:52:16.122 --> 03:52:17.556
A COMPUTER EXPERT AT ALL, BUT

03:52:17.623 --> 03:52:19.325
WE HEAR A LOT ABOUT QUANTUM

03:52:19.392 --> 03:52:20.760
COMPUTING AND WONDERING WHETHER

03:52:20.826 --> 03:52:22.495
THAT WOULD ALSO CHANGE OUR

03:52:22.561 --> 03:52:24.330
UNDERSTANDING WITHIN CAN CAN

03:52:24.397 --> 03:52:25.464
QUANTUM COMPUTING BE APPLIED TO

03:52:25.531 --> 03:52:27.933
THE AI SPACE? AND WILL THAT

03:52:28.000 --> 03:52:29.402
CHANGE THE WAY WE MEASURE

03:52:29.468 --> 03:52:32.672
COMPUTE POWER? I MAYBE I'LL

03:52:32.738 --> 03:52:36.542
FIELD THIS ONE. SO I THINK THE

03:52:36.609 --> 03:52:38.377
MOST USEFUL WAY TO THINK ABOUT

03:52:38.444 --> 03:52:39.845
QUANTUM COMPUTING VERSUS AI IS

03:52:39.912 --> 03:52:41.580
THAT THEY ARE JUST DIFFERENT

03:52:41.647 --> 03:52:43.816
TECHNOLOGIES. AND QUANTUM

03:52:43.883 --> 03:52:46.652
COMPUTING IS VERY EARLY AND,

03:52:46.719 --> 03:52:49.188
YOU KNOW, KIND OF A LONG TERM

03:52:49.255 --> 03:52:50.990
RESEARCH PROJECT. AND I THINK

03:52:51.057 --> 03:52:52.858
IT IS COMPLETELY POSSIBLE THAT

03:52:52.925 --> 03:52:54.593
AT SOME POINT THEY'RE GOING TO

03:52:54.660 --> 03:52:56.028
INTERACT IN KIND OF INTERESTING

03:52:56.095 --> 03:52:58.964
WAYS. BUT FOR NOW, I THINK THAT

03:52:59.031 --> 03:53:00.700
THE ONLY INTERACTION IS THAT

03:53:00.766 --> 03:53:02.501
MAYBE AI CODING AGENTS WILL

03:53:02.568 --> 03:53:03.769
HELP SPEED UP QUANTUM COMPUTING,

03:53:03.836 --> 03:53:05.738
BUT I DON'T EXPECT ANY

03:53:05.805 --> 03:53:07.206
SIGNIFICANT IMPACT ON QUANTUM

03:53:07.273 --> 03:53:10.076
COMPUTING, ON AI DEVELOPMENT OR

03:53:10.142 --> 03:53:12.211
AI SAFETY ANYTIME SOON, OR

03:53:12.278 --> 03:53:14.747
CAPABILITY OR CAPABILITY. OKAY.

03:53:14.814 --> 03:53:15.881
THANK YOU. AND THEN MY LAST

03:53:15.948 --> 03:53:17.049
QUESTION IS MORE, GETTING BACK

03:53:17.116 --> 03:53:18.718
TO THE CYBERSECURITY, KIND OF

03:53:18.784 --> 03:53:19.819
THE MORE NUTS AND BOLTS

03:53:19.885 --> 03:53:23.022
NEAR-TERM WORRIES, HOW WOULD

03:53:23.089 --> 03:53:24.824
YOU ADVISE THE STATES TO

03:53:24.890 --> 03:53:26.592
PROTECT THEMSELVES, THE STATE,

03:53:26.659 --> 03:53:27.993
SPECIFICALLY IN OUR AGENCIES

03:53:28.060 --> 03:53:29.462
AND DEPARTMENTS TO PROTECT

03:53:29.528 --> 03:53:31.097
AGAINST CYBER SECURITY ATTACKS?

03:53:31.163 --> 03:53:32.465
WE ALREADY WORRY ABOUT THEM

03:53:32.531 --> 03:53:34.900
FROM REGULAR NON AI BASED

03:53:34.967 --> 03:53:37.036
ACTORS. WE RECENTLY HAD A HACK

03:53:37.103 --> 03:53:38.838
OF OUR DATA SYSTEM WITH OUR

03:53:38.904 --> 03:53:41.006
STUDENT DATA AND JUST WONDERING,

03:53:41.073 --> 03:53:43.542
DO YOU HAVE ANY SUGGESTIONS FOR

03:53:43.609 --> 03:53:45.311
HOW WE CAN CONTINUE TO BE, YOU

03:53:45.378 --> 03:53:46.746
KNOW, TO HAVE THAT EARLY

03:53:46.812 --> 03:53:48.214
DEPLOYMENT TO THE DEFENDERS

03:53:48.280 --> 03:53:50.015
FIRST? HOW DO WE ENGAGE IN THAT

03:53:50.082 --> 03:53:52.184
KIND OF SPACE? YEAH. SO I DON'T

03:53:52.251 --> 03:53:53.652
NORMALLY LIKE TO BE A POLICY

03:53:53.719 --> 03:53:55.221
PERSON PLUGGING OUR PRODUCTS.

03:53:55.287 --> 03:53:57.189
BUT IN THIS CASE, OUR CYBER

03:53:57.256 --> 03:53:59.859
DEFENDER TOOLS ARE ARE VERY,

03:53:59.925 --> 03:54:01.794
VERY USEFUL. AND WE WORK WITH A

03:54:01.861 --> 03:54:02.962
LOT OF PUBLIC SECTOR ACTORS

03:54:03.028 --> 03:54:05.898
AROUND THE WORLD TO, TO GET

03:54:05.965 --> 03:54:08.367
THEM THOSE TOOLS TO, TO DO THAT

03:54:08.434 --> 03:54:10.202
WORK. WE'RE IN A PHASE RIGHT

03:54:10.269 --> 03:54:13.773
NOW WHERE WE ARE DEALING WITH

03:54:13.839 --> 03:54:17.410
THE LEGACY OF HUMAN CREATED

03:54:17.476 --> 03:54:18.944
SOFTWARE OVER TENS OF YEARS,

03:54:19.011 --> 03:54:20.212
AND THERE'S LOTS AND LOTS OF

03:54:20.279 --> 03:54:21.313
VULNERABILITIES IN THAT

03:54:21.380 --> 03:54:23.816
SOFTWARE. IT'S A FINITE AMOUNT.

03:54:23.883 --> 03:54:25.651
AND SO THOSE VULNERABILITIES

03:54:25.718 --> 03:54:27.119
CAN ALL EVENTUALLY BE

03:54:27.186 --> 03:54:28.487
DISCOVERED AND PATCHED. AND SO

03:54:28.554 --> 03:54:30.389
USING THOSE AI PATCHING TOOLS

03:54:30.456 --> 03:54:32.925
IS THE WAY AHEAD. AND THERE IS

03:54:32.992 --> 03:54:34.593
A LIGHT AT THE END OF THAT

03:54:34.660 --> 03:54:36.529
TUNNEL. WE JUST HAVE TO DO THAT

03:54:36.595 --> 03:54:37.897
HARD WORK. AND SO I KNOW FROM

03:54:37.963 --> 03:54:39.732
THE GOOGLE AND GOOGLE CLOUD

03:54:39.799 --> 03:54:41.634
PERSPECTIVE, WE HAVE PROGRAMS

03:54:41.700 --> 03:54:43.669
TO WORK WITH THE PUBLIC SECTOR

03:54:43.736 --> 03:54:45.805
TO SHORE UP THOSE SYSTEMS. I

03:54:45.871 --> 03:54:46.872
HATE TO DISAGREE WITH MY

03:54:46.939 --> 03:54:49.809
COLLEAGUE HERE, BUT WE WERE

03:54:49.875 --> 03:54:53.746
QUITE A BIT ON CYBERSECURITY,

03:54:53.813 --> 03:54:55.614
AND I AM NOT CONFIDENT THAT

03:54:55.681 --> 03:54:57.183
THERE IS A FINITE NUMBER OF

03:54:57.249 --> 03:54:58.350
BUGS THAT YOU WILL BE ABLE TO

03:54:58.417 --> 03:55:00.252
JUST PATCH. UNFORTUNATELY, CODE

03:55:00.319 --> 03:55:03.422
IS ALWAYS EVOLVING AND CHANGING,

03:55:03.489 --> 03:55:04.657
AND IT'S NOT CLEAR TO ME THAT

03:55:04.723 --> 03:55:06.559
WE'LL BE ABLE TO PATCH OUR WAY

03:55:06.625 --> 03:55:08.527
OUT OF THIS. NOW, IN THE SHORT

03:55:08.594 --> 03:55:10.696
TERM, I 100% AGREE THAT THE

03:55:10.763 --> 03:55:12.097
AGENCY SHOULD SIGN UP WITH ONE

03:55:12.164 --> 03:55:13.732
OF THE FRONTIER LABS, OR ALL

03:55:13.799 --> 03:55:15.734
THREE OF THEM IMPROVE THE

03:55:15.801 --> 03:55:17.837
CYBERSECURITY OF THEIR CODE.

03:55:17.903 --> 03:55:19.305
YOU'RE A LITTLE BIT IN A RACE

03:55:19.371 --> 03:55:21.474
WITH OPEN SOURCE CAPABILITIES

03:55:21.540 --> 03:55:22.875
WHERE, YOU KNOW, THEY'RE

03:55:22.942 --> 03:55:24.043
INCREASING EVERY DAY. AND SO

03:55:24.109 --> 03:55:26.111
THE WORST CYBER ACTORS ON THE

03:55:26.178 --> 03:55:27.179
PLANET ARE BECOMING MORE

03:55:27.246 --> 03:55:29.748
CAPABLE EVERY DAY. AND YOU HAVE

03:55:29.815 --> 03:55:31.884
TO SHORE UP YOUR DEFENSES

03:55:31.951 --> 03:55:35.588
BEFORE THEN. BUT I ALSO THINK

03:55:35.654 --> 03:55:37.990
THAT WE ARE GOING TO NEED A

03:55:38.057 --> 03:55:40.493
MORE SUBSTANTIAL EFFORT TO

03:55:40.559 --> 03:55:41.594
FUNDAMENTALLY REWRITE THE

03:55:41.660 --> 03:55:44.096
UNDERPINNINGS OF THE INTERNET

03:55:44.163 --> 03:55:46.198
INTO, YOU KNOW, SYSTEMS THAT

03:55:46.265 --> 03:55:48.801
ARE JUST MUCH MORE SECURE.

03:55:48.868 --> 03:55:50.302
WE'LL NEED TO MOVE AWAY FROM

03:55:50.369 --> 03:55:51.670
THE COMMON LANGUAGES THAT ARE

03:55:51.737 --> 03:55:55.674
NOT MEMORY SAFE AND MOVE TO

03:55:55.741 --> 03:55:57.810
PROVABLY, VERIFIABLY CORRECT

03:55:57.877 --> 03:55:59.245
IMPLEMENTATIONS OF, YOU KNOW,

03:55:59.311 --> 03:56:00.713
THE FUNDAMENTAL UNDERPINNINGS

03:56:00.779 --> 03:56:03.282
OF THE INTERNET. AND SO, YOU

03:56:03.349 --> 03:56:05.017
KNOW, IN THE SHORT TERM, PLEASE

03:56:05.084 --> 03:56:07.119
HAVE ALL OF YOUR AGENCIES, YOU

03:56:07.186 --> 03:56:08.954
KNOW, WORK HARD TO USE AI TO

03:56:09.021 --> 03:56:11.457
PATCH THEIR CODE. AND I THINK,

03:56:11.524 --> 03:56:13.626
YOU KNOW, IT WOULD PROBABLY BE

03:56:13.692 --> 03:56:14.760
HELPFUL FOR CALIFORNIA TO TAKE

03:56:14.827 --> 03:56:17.229
A LEADERSHIP ROLE IN THIS MAJOR

03:56:17.296 --> 03:56:19.365
PROJECT. YOU KNOW, WE, YOU KNOW,

03:56:19.431 --> 03:56:21.934
INVENTED THE INTERNET. AND I

03:56:22.001 --> 03:56:22.801
THINK WE'RE GOING TO HAVE TO

03:56:22.868 --> 03:56:24.803
INVENT THE NEXT ONE TO GET TO A

03:56:24.870 --> 03:56:26.639
GOOD PLACE IN CYBER SECURITY.

03:56:26.705 --> 03:56:29.174
WOW. INTERNET 2.0 MASSIVE

03:56:29.241 --> 03:56:30.175
UNDERTAKING. THAT'S I THINK

03:56:30.242 --> 03:56:31.277
THAT BRAND IS TAKEN. SO MAYBE

03:56:31.343 --> 03:56:34.146
WE NEED ANOTHER ONE. BUT. YES,

03:56:34.213 --> 03:56:37.049
EXACTLY. AND I WILL JUST JUMP

03:56:37.116 --> 03:56:39.418
IN QUICKLY TO SAY THE BLUEPRINT

03:56:39.485 --> 03:56:40.352
THAT I MENTIONED THAT WE

03:56:40.419 --> 03:56:41.487
RELEASED LAST WEEK ACTUALLY

03:56:41.554 --> 03:56:42.555
ADDRESSES EXACTLY THIS POINT,

03:56:42.621 --> 03:56:43.889
NOT JUST WHAT FRONTIER

03:56:43.956 --> 03:56:45.724
DEVELOPERS CAN DO, BUT WHAT

03:56:45.791 --> 03:56:47.126
STATE, LOCAL AND FEDERAL

03:56:47.192 --> 03:56:48.227
GOVERNMENT SHOULD BE DOING TO

03:56:48.294 --> 03:56:50.396
STRENGTHEN CYBERSECURITY. YOU

03:56:50.462 --> 03:56:52.231
KNOW, I THINK AS, AS DOCTOR

03:56:52.298 --> 03:56:54.700
FRIEND SAID, THERE IS A LOT OF

03:56:54.767 --> 03:56:56.869
TECHNICAL DEBT THAT EXISTS IN

03:56:56.936 --> 03:56:57.870
ORGANIZATIONS THAT ARE JUST

03:56:57.937 --> 03:57:00.439
THEY'RE OPERATING VULNERABLE

03:57:00.506 --> 03:57:02.207
SYSTEMS. THERE'S LOTS OF

03:57:02.274 --> 03:57:03.309
OPPORTUNITIES FOR PUBLIC

03:57:03.375 --> 03:57:06.312
PRIVATE PARTNERSHIPS. HAVING

03:57:06.378 --> 03:57:08.113
FRONTIER AI LABS WORKING WITH

03:57:08.180 --> 03:57:09.882
PUBLIC SECTOR ORGANIZATIONS TO

03:57:09.949 --> 03:57:12.051
HELP THEM USE OUR TOOLS TO

03:57:12.117 --> 03:57:12.885
IDENTIFY VULNERABILITIES. BUT

03:57:12.952 --> 03:57:14.820
IN SOME CASES, YOU STILL NEED

03:57:14.887 --> 03:57:17.222
TO MAKE INVESTMENTS IN HARDWARE.

03:57:17.289 --> 03:57:18.824
AND IN ALMOST ALL CASES, YOU

03:57:18.891 --> 03:57:20.693
NEED TO MAKE INVESTMENTS IN THE

03:57:20.759 --> 03:57:22.094
WORKFORCE AND THE CYBERSECURITY

03:57:22.161 --> 03:57:24.296
WORKFORCE TO UPSKILL PEOPLE. SO

03:57:24.363 --> 03:57:25.731
OPENAI IS PROUD TO DO A LOT OF

03:57:25.798 --> 03:57:27.600
A LOT OF THAT WORK. WE'RE

03:57:27.666 --> 03:57:29.702
PARTNERING WITH CAL OES. IT WAS

03:57:29.768 --> 03:57:32.905
ACTUALLY OUR PATCH THE PLANET

03:57:32.972 --> 03:57:34.273
PROGRAM LAST WEEK THAT

03:57:34.340 --> 03:57:36.475
IDENTIFIED THE VULNERABILITY

03:57:36.542 --> 03:57:38.711
THAT I THINK UNDERPINNED LIKE

03:57:38.777 --> 03:57:40.112
60% OF THE INTERNET. AND IT WAS

03:57:40.179 --> 03:57:43.649
THROUGH THAT PROGRAM THAT WE

03:57:43.716 --> 03:57:46.585
LEAD, THAT WE COORDINATED THE

03:57:46.652 --> 03:57:48.020
IDENTIFICATION OF THAT PATCH OR

03:57:48.087 --> 03:57:49.788
THE VULNERABILITY AND THEN THE

03:57:49.855 --> 03:57:50.856
DEPLOYMENT OF THAT PATCH. SO

03:57:50.923 --> 03:57:52.024
WE'RE GOING TO CONTINUE THAT

03:57:52.091 --> 03:57:53.792
WORK, BUT MORE ACTION IS NEEDED.

03:57:53.859 --> 03:57:54.994
WE'LL SEND YOU THE BLUEPRINT OF

03:57:55.060 --> 03:57:56.795
IDEAS. THANK YOU. WE'RE FINDING

03:57:56.862 --> 03:57:58.030
WHERE WE'RE GOING TO INCREASE

03:57:58.097 --> 03:58:00.399
THE WORKFORCE. I LIKE IT AGAIN

03:58:00.466 --> 03:58:01.600
NOT THE SUBJECT OF THE HEARING,

03:58:01.667 --> 03:58:05.638
BUT IMPORTANT SIDE NOTES. SO I

03:58:05.704 --> 03:58:07.840
HAVE A QUESTION. I KNOW YOU

03:58:07.906 --> 03:58:08.974
PROBABLY DIDN'T PLAN LAST WEEK

03:58:09.041 --> 03:58:10.609
WHEN YOU WALKED IN. MAYBE YOU

03:58:10.676 --> 03:58:11.744
KNEW YOUR IPO DOCUMENT WAS

03:58:11.810 --> 03:58:13.545
GOING TO DROP THE DAY BEFORE.

03:58:13.612 --> 03:58:16.415
NOPE. NO. SO IT WAS OPENAI HAS

03:58:16.482 --> 03:58:18.117
A LITTLE BIT OF AN ADVANTAGE

03:58:18.183 --> 03:58:19.151
THAT THEY'RE NOT PUBLICLY

03:58:19.218 --> 03:58:20.285
TRADED, NOR HAVE THEY FILED AN

03:58:20.352 --> 03:58:22.955
OPEN IPO DOCUMENT. AND I

03:58:23.022 --> 03:58:24.690
HAVEN'T ACTUALLY LOOKED AT YOUR

03:58:24.757 --> 03:58:25.758
SEC DOCUMENTS TO SEE IF IT

03:58:25.824 --> 03:58:27.660
TALKS ABOUT AI RISKS. BUT YOUR

03:58:27.726 --> 03:58:29.428
YOUR IPO OBVIOUSLY SPENT. I

03:58:29.495 --> 03:58:30.863
WANT TO SAY ALMOST A THIRD OF

03:58:30.929 --> 03:58:32.264
THE DOCUMENT TALKING ABOUT AI

03:58:32.331 --> 03:58:37.469
RISKS. AND ONE OF THE THINGS

03:58:37.536 --> 03:58:39.505
THAT APPEARED THAT I UNDERSTOOD

03:58:39.571 --> 03:58:40.706
FROM IT WAS THAT AT ONE POINT,

03:58:40.773 --> 03:58:44.176
YOU SENT 6% OF YOUR RESEARCH

03:58:44.243 --> 03:58:47.112
CAPACITY ON SAFETY. THAT FELT A

03:58:47.179 --> 03:58:49.181
LITTLE BIT LOPSIDED TO ME, BUT

03:58:49.248 --> 03:58:50.983
MAYBE THAT WAS I THINK THAT WAS

03:58:51.050 --> 03:58:52.418
JUST ONE MONTH FROM WHAT I

03:58:52.484 --> 03:58:54.253
FOUND. BUT IT SORT OF RAISED

03:58:54.319 --> 03:58:55.354
THE QUESTION OF HOW MUCH OF

03:58:55.421 --> 03:58:57.156
YOUR RESOURCES ARE GOING TO

03:58:57.222 --> 03:59:00.693
SAFETY. I'LL ASK EACH OF YOU,

03:59:00.759 --> 03:59:03.162
DO YOU KNOW I'LL HAVE TO GET

03:59:03.228 --> 03:59:04.430
BACK TO YOU ON THAT QUESTION?

03:59:04.496 --> 03:59:06.465
OKAY. YEAH, I DON'T KNOW A

03:59:06.532 --> 03:59:07.733
RESOURCE NUMBER. I KNOW THAT'S

03:59:07.800 --> 03:59:10.936
AROUND A THOUSAND PEOPLE. CORE

03:59:11.003 --> 03:59:14.006
SAFETY ROLES ON AI OR ON OKAY,

03:59:14.073 --> 03:59:16.208
I DON'T KNOW THE NUMBER EITHER.

03:59:16.275 --> 03:59:17.776
IF YOU WANTED MORE DETAILS, YOU

03:59:17.843 --> 03:59:19.111
COULD ALSO THINK ABOUT COMPUTE

03:59:19.178 --> 03:59:20.345
VERSUS PEOPLE BECAUSE THOSE

03:59:20.412 --> 03:59:22.347
THINGS MIGHT BE DIFFERENT. TRUE.

03:59:22.414 --> 03:59:23.682
AND I DO THINK, I MEAN, I

03:59:23.749 --> 03:59:24.683
APPRECIATED SOME OF THE

03:59:24.750 --> 03:59:25.784
QUESTIONS THAT WERE ASKED ABOUT

03:59:25.851 --> 03:59:28.120
RESEARCH. I THINK AS I HAVE HAD

03:59:28.187 --> 03:59:28.954
CONVERSATIONS WITH MANY PEOPLE

03:59:29.021 --> 03:59:31.356
OVER INTERIM, I AM DOUBLING

03:59:31.423 --> 03:59:33.158
DOWN ON CAL COMPUTE AS THE

03:59:33.225 --> 03:59:34.960
THING THAT THE SAFETY MECHANISM

03:59:35.027 --> 03:59:35.961
NOBODY IS TALKING ABOUT,

03:59:36.028 --> 03:59:37.429
BECAUSE I DO ACTUALLY BELIEVE,

03:59:37.496 --> 03:59:38.797
AND I THINK YOU SAID THIS,

03:59:38.864 --> 03:59:41.066
DOCTOR DWYER, THAT YOU

03:59:41.133 --> 03:59:42.534
SHOULDN'T BE THE ONLY ONE DOING

03:59:42.601 --> 03:59:43.635
THE RESEARCH ON SAFETY. AND

03:59:43.702 --> 03:59:46.438
IT'S THE TRUTH. AND WE ARE

03:59:46.505 --> 03:59:47.573
UNIVERSITIES AND OUR ACADEMICS

03:59:47.639 --> 03:59:48.974
WHO COULD BE DOING SOME OF THE

03:59:49.041 --> 03:59:50.409
MOST IMPORTANT RESEARCH,

03:59:50.476 --> 03:59:51.777
WHETHER IT'S HOW DO YOU HAVE A

03:59:51.844 --> 03:59:53.679
REAL KILL SWITCH? HOW DO YOU,

03:59:53.746 --> 03:59:55.414
YOU KNOW, CREATE SAFETY IN A

03:59:55.481 --> 03:59:57.216
MORE, YOU KNOW, HOW DO WE DEAL

03:59:57.282 --> 03:59:58.417
WITH SOME OF THE CYBERSECURITY

03:59:58.484 --> 03:59:59.318
THREATS WE DON'T EVEN

03:59:59.384 --> 04:00:01.153
UNDERSTAND YET IS GOING TO COME

04:00:01.220 --> 04:00:02.154
OUT OF OUR ACADEMIC

04:00:02.221 --> 04:00:03.122
INSTITUTIONS. AND RIGHT NOW,

04:00:03.188 --> 04:00:04.356
THEY DON'T HAVE THE CAPACITY TO

04:00:04.423 --> 04:00:06.158
DO THAT BECAUSE THEY DON'T HAVE

04:00:06.225 --> 04:00:08.393
THE COMPUTE POWER. AND SO I DO

04:00:08.460 --> 04:00:09.528
ACTUALLY THINK THAT WE NEED TO

04:00:09.595 --> 04:00:10.963
BE INVESTING, AND HOPEFULLY WE

04:00:11.029 --> 04:00:13.799
CAN DO THAT IN A GLOBAL WAY IN

04:00:13.866 --> 04:00:15.634
PARTNERSHIP WITH YOUR LABS, TO

04:00:15.701 --> 04:00:16.969
INVEST IN THE ABILITY FOR

04:00:17.035 --> 04:00:18.170
ACADEMICS TO BE IN THIS

04:00:18.237 --> 04:00:19.138
DISCUSSION AS WELL. I THINK IT

04:00:19.204 --> 04:00:20.939
WOULD BENEFIT EACH AND EVERY

04:00:21.006 --> 04:00:22.107
ONE OF YOUR COMPANIES TO HAVE

04:00:22.174 --> 04:00:23.942
OUR ACADEMICS IN THIS SPACE AS

04:00:24.009 --> 04:00:25.344
WELL. SO I, I THINK RESEARCH IS

04:00:25.410 --> 04:00:26.945
CRITICAL, AND I THINK I

04:00:27.012 --> 04:00:28.213
APPRECIATE YOU DOING IT. I ALSO

04:00:28.280 --> 04:00:29.948
THINK WE NEED TO BE GETTING

04:00:30.015 --> 04:00:31.216
INTO THE GAME A LITTLE BIT MORE

04:00:31.283 --> 04:00:34.653
AS WELL. THE OTHER THING THAT

04:00:34.720 --> 04:00:36.522
WAS INTERESTING TO ME ABOUT THE

04:00:36.588 --> 04:00:39.725
IPO DOCUMENT WAS THAT IT SEEMED

04:00:39.792 --> 04:00:42.594
TO IMPLY THAT RECURSIVE

04:00:42.661 --> 04:00:44.429
SELF-IMPROVEMENT WAS NECESSARY

04:00:44.496 --> 04:00:45.998
FOR THE SUCCESS OF THE COMPANY.

04:00:46.064 --> 04:00:47.466
I KNOW WE HAVEN'T GOTTEN TO

04:00:47.533 --> 04:00:50.202
RECURSIVE SELF-IMPROVEMENT, BUT

04:00:50.269 --> 04:00:51.370
IT ALSO SEEMED TO IMPLY THAT

04:00:51.436 --> 04:00:52.671
ONCE WE GOT TO RECURSIVE

04:00:52.738 --> 04:00:53.739
SELF-IMPROVEMENT, IT WAS

04:00:53.806 --> 04:00:55.941
UNCLEAR IF YOU BELIEVED THAT WE

04:00:56.008 --> 04:00:58.076
COULD KEEP UP WITH A SAFETY

04:00:58.143 --> 04:00:58.911
PERSPECTIVE. BECAUSE ALL OF

04:00:58.977 --> 04:01:00.145
WHAT YOU'VE TALKED ABOUT TODAY,

04:01:00.212 --> 04:01:02.848
THE MANY LAYERS, THE WONDERFUL

04:01:02.915 --> 04:01:04.850
WORK YOU'RE DOING ONCE WE GET

04:01:04.917 --> 04:01:07.052
TO RECURSIVE SELF-IMPROVEMENT,

04:01:07.119 --> 04:01:08.554
WILL YOU HAVE THE ABILITY TO DO

04:01:08.620 --> 04:01:11.056
THOSE THINGS AS THE MODEL

04:01:11.123 --> 04:01:15.327
IMPROVES ITSELF AT A RAPID PACE?

04:01:15.394 --> 04:01:16.795
I DON'T THINK WE SHOULD START

04:01:16.862 --> 04:01:18.230
RECURSIVE SELF-IMPROVEMENT

04:01:18.297 --> 04:01:21.099
UNTIL WE ARE CONFIDENT THAT THE

04:01:21.166 --> 04:01:23.902
SAFETY TECHNOLOGY IS READY AND

04:01:23.969 --> 04:01:25.470
OUR ALIGNMENT TECHNOLOGY IS

04:01:25.537 --> 04:01:26.839
READY, AND THAT IS NOT TRUE

04:01:26.905 --> 04:01:29.041
TODAY. DO YOU THINK YOU CAN

04:01:29.107 --> 04:01:30.108
FORESTALL RECURSIVE

04:01:30.175 --> 04:01:36.982
SELF-IMPROVEMENT? THE BASIC

04:01:37.049 --> 04:01:38.750
IDEA BEHIND THE PACING, THE

04:01:38.817 --> 04:01:41.653
FRONTIER ESSAY WAS TO SAY, YOU

04:01:41.720 --> 04:01:44.089
KNOW, DON'T MOVE FORWARD UNTIL

04:01:44.156 --> 04:01:46.358
THE SAFETY TECHNOLOGY IS READY

04:01:46.425 --> 04:01:49.862
FOR ANY GIVEN CAPABILITIES GAIN.

04:01:49.928 --> 04:01:52.397
AND THAT WAS WRITTEN A, YOU

04:01:52.464 --> 04:01:55.634
KNOW, A CERTAINLY HAD RSI IN

04:01:55.701 --> 04:01:58.136
MIND. SO, YOU KNOW, I THINK

04:01:58.203 --> 04:01:59.671
THAT THE IDEA BEHIND PACING THE

04:01:59.738 --> 04:02:01.206
FRONTIER IS THAT IF YOU ARE NOT

04:02:01.273 --> 04:02:03.609
READY FOR RSI, IF YOUR SAFETY

04:02:03.675 --> 04:02:04.643
TECHNOLOGIES ARE NOT THERE, IF

04:02:04.710 --> 04:02:06.445
YOU DON'T HAVE A GOOD SAFETY

04:02:06.511 --> 04:02:09.348
CASE, THEN YOU SHOULD NOT DO IT.

04:02:09.414 --> 04:02:12.351
AND I APPRECIATED ANTHROPIC PUT

04:02:12.417 --> 04:02:14.052
OUT A PAPER JUST THIS WEEK. ONE

04:02:14.119 --> 04:02:15.487
OF THE THINGS WE HAD BELIEVED

04:02:15.554 --> 04:02:18.724
WAS GOING TO STOP THE

04:02:18.790 --> 04:02:20.926
EXFILTRATION THREAT WAS THE

04:02:20.993 --> 04:02:22.127
NEED FOR DATA CENTERS. AND THEN

04:02:22.194 --> 04:02:23.762
THE PAPER THAT SOMEONE AT

04:02:23.829 --> 04:02:24.830
ANTHROPIC PUT OUT THIS WEEK

04:02:24.897 --> 04:02:25.898
SAYING, MAYBE IT'S POSSIBLE

04:02:25.964 --> 04:02:28.400
WITHOUT THAT, AND THEY CAN DO

04:02:28.467 --> 04:02:30.969
IT ON TOASTERS. I'M SUMMARIZING

04:02:31.036 --> 04:02:32.004
IN A WAY THAT'S PROBABLY

04:02:32.070 --> 04:02:34.206
FLIPPANT, BUT, YOU KNOW, I DO

04:02:34.273 --> 04:02:36.341
THINK THIS IS I APPRECIATE THE

04:02:36.408 --> 04:02:37.342
TRANSPARENCY AND THE AWARENESS

04:02:37.409 --> 04:02:39.611
YOU'RE BRINGING TO IT BECAUSE I

04:02:39.678 --> 04:02:41.380
WANT TO MAKE SURE THAT WE DO

04:02:41.446 --> 04:02:42.714
ACTUALLY, WE ARE ABLE TO

04:02:42.781 --> 04:02:43.916
FORESTALL IT IF WE'RE NOT READY

04:02:43.982 --> 04:02:44.950
FOR IT FROM A SAFETY

04:02:45.017 --> 04:02:47.085
PERSPECTIVE. ANY THOUGHTS? I

04:02:47.152 --> 04:02:48.353
WOULD JUST SAY THAT, YOU KNOW,

04:02:48.420 --> 04:02:50.756
WE SHOULDN'T BE TRAINING MODELS

04:02:50.822 --> 04:02:52.457
IF WE CAN'T BE SURE THAT WE CAN

04:02:52.524 --> 04:02:54.393
KEEP THEM UNDER CONTROL. AND SO

04:02:54.459 --> 04:02:55.827
I THINK THAT IS THE CORE IDEA

04:02:55.894 --> 04:02:58.330
BEHIND THAT PACING, THE

04:02:58.397 --> 04:03:00.532
FRONTIER STATEMENT AND IS, IS

04:03:00.599 --> 04:03:02.000
CORE TO HOW WE DEVELOP MODELS.

04:03:02.067 --> 04:03:03.135
OKAY. AND WE'LL SEE THAT PAPER

04:03:03.201 --> 04:03:04.937
COMING OUT SOON. THOSE WILL BE

04:03:05.003 --> 04:03:06.071
POLICY IDEAS, NOT TECHNICAL

04:03:06.138 --> 04:03:07.906
ONES. BUT YOU KNOW WHAT I MEAN.

04:03:07.973 --> 04:03:09.441
THEY'LL BE REALLY GOOD. I

04:03:09.508 --> 04:03:11.009
ADVERTISE YOUR PAPER AGAIN,

04:03:11.076 --> 04:03:12.311
DOCTOR FRIEND. YEAH. AND THAT'S

04:03:12.377 --> 04:03:13.345
THE BASIC STATEMENT IN OUR

04:03:13.412 --> 04:03:14.513
FRONTIER SAFETY FRAMEWORK. AS I

04:03:14.579 --> 04:03:17.049
MENTIONED EARLIER, WE ADDRESS

04:03:17.115 --> 04:03:18.784
MODEL R&D DIRECTLY IN THERE.

04:03:18.850 --> 04:03:21.019
AND WE ALSO BELIEVE SIMILARLY

04:03:21.086 --> 04:03:22.754
THAT WE SHOULDN'T WE SHOULDN'T

04:03:22.821 --> 04:03:24.423
LOSE CONTROL OF THESE SYSTEMS.

04:03:24.489 --> 04:03:25.490
AND WE SHOULD CONTINUE TO

04:03:25.557 --> 04:03:26.758
ENGINEER CONTROL INTO THE

04:03:26.825 --> 04:03:28.327
DEVELOPMENT PROCESS. AND THEN

04:03:28.393 --> 04:03:31.096
MY LAST QUESTION, SO AND THIS

04:03:31.163 --> 04:03:32.331
ONE IS SPECIFICALLY FOR YOU,

04:03:32.397 --> 04:03:34.366
DOCTOR. SO ONE OF YOUR TESTS

04:03:34.433 --> 04:03:35.734
SHOWED YOU HOW TO MODEL THAT

04:03:35.801 --> 04:03:37.536
WENT OUT AND ACTUALLY TURNED

04:03:37.602 --> 04:03:38.937
AROUND WHEN IT WENT TO DO

04:03:39.004 --> 04:03:40.072
SOMETHING THAT WAS MISALIGNED,

04:03:40.138 --> 04:03:42.007
STOPPED, STOPPED, TURNED AROUND.

04:03:42.074 --> 04:03:42.975
THEY DON'T THEY'RE NOT LIKE

04:03:43.041 --> 04:03:44.376
THEY'RE, THEY'RE SENDING PULSES

04:03:44.443 --> 04:03:45.444
INTO THE INTERNET. YEAH.

04:03:45.510 --> 04:03:48.714
STOPPED. WE HAVE SEEN AND I

04:03:48.780 --> 04:03:50.349
DON'T KNOW IF EITHER OF YOU

04:03:50.415 --> 04:03:51.450
HAVE EXAMPLES WHERE THAT IS THE

04:03:51.516 --> 04:03:53.485
CASE. YES YOU DO. SO WHAT IS IT

04:03:53.552 --> 04:03:55.921
THAT MAKES. WHY DOES THAT

04:03:55.988 --> 04:03:57.689
HAPPEN. AND THEN WE HAVE I MEAN,

04:03:57.756 --> 04:03:59.124
THE ALIGNMENT IS WORKING SOME

04:03:59.191 --> 04:04:00.258
OF THE TIME, BUT IT'S NOT

04:04:00.325 --> 04:04:02.160
WORKING ALL OF THE TIME. DO WE

04:04:02.227 --> 04:04:04.563
KNOW WHY IT IS WORKING? WHY

04:04:04.629 --> 04:04:06.031
QUESTIONS ARE VERY DIFFICULT TO

04:04:06.098 --> 04:04:11.003
LIKE COMPLETELY NAIL DOWN. BUT

04:04:11.069 --> 04:04:13.238
WE RECENTLY RELEASED A LAST

04:04:13.305 --> 04:04:16.408
WEEK, MAYBE A KIND OF DEEP

04:04:16.475 --> 04:04:17.442
INVESTIGATION OF THE INCIDENT

04:04:17.509 --> 04:04:18.977
THAT HAPPENED WITH UK ATC WHERE

04:04:19.044 --> 04:04:22.147
OUR MODEL, YOU KNOW, DID SOME

04:04:22.214 --> 04:04:23.148
INAPPROPRIATE THINGS AND TRYING

04:04:23.215 --> 04:04:24.483
TO UPLOAD A MALICIOUS PACKAGE

04:04:24.549 --> 04:04:27.185
TO GITHUB. AND ONE OF THE

04:04:27.252 --> 04:04:28.320
THINGS THEY FOUND IS THAT THE

04:04:28.387 --> 04:04:30.522
MODEL SORT OF HAD A KIND OF

04:04:30.589 --> 04:04:32.657
MOMENTUM TO IT. AND YOU SEE

04:04:32.724 --> 04:04:35.494
THIS WITH PEOPLE TOO, WHERE,

04:04:35.560 --> 04:04:37.362
YOU KNOW, YOU KEEP GOING DOWN A

04:04:37.429 --> 04:04:38.764
PATH AND YOU CAN KIND OF LIKE

04:04:38.830 --> 04:04:40.565
NOT REMEMBER TO STOP AND TAKE A

04:04:40.632 --> 04:04:41.733
LOOK AROUND AND REALIZE THAT

04:04:41.800 --> 04:04:43.535
THE PATH YOU'RE GOING DOWN IS

04:04:43.602 --> 04:04:44.736
MAYBE NOT THE RIGHT ONE. AND SO

04:04:44.803 --> 04:04:46.638
THE MODELS, IF YOU PUT THEM IN

04:04:46.705 --> 04:04:48.540
A SITUATION WHERE THEY HAD THIS

04:04:48.607 --> 04:04:49.908
OPPORTUNITY, WOULD USUALLY

04:04:49.975 --> 04:04:52.110
REFUSE IT. BUT OCCASIONALLY, IF

04:04:52.177 --> 04:04:53.245
THEY HAD KIND OF TALKED

04:04:53.311 --> 04:04:54.312
THEMSELVES INTO THINKING THAT

04:04:54.379 --> 04:04:55.380
THE NEXT STEP WAS REASONABLE

04:04:55.447 --> 04:04:57.149
AND THEN TOOK THAT STEP AND

04:04:57.215 --> 04:04:58.250
THEN, YOU KNOW, KIND OF KEPT

04:04:58.316 --> 04:05:00.118
GOING DOWN THAT PATH WOULD KEEP

04:05:00.185 --> 04:05:02.154
GOING, EVEN WHEN, YOU KNOW, IT

04:05:02.220 --> 04:05:03.355
GOT TO THE POINT WHERE THEY

04:05:03.422 --> 04:05:04.389
CLEARLY SHOULD HAVE KNOWN

04:05:04.456 --> 04:05:06.525
BETTER. AND I GUESS I'LL SAY A

04:05:06.591 --> 04:05:09.694
COUPLE THINGS ABOUT THIS. ONE

04:05:09.761 --> 04:05:13.331
IS JUST THE EXPERIENCE OF BEING

04:05:13.398 --> 04:05:15.000
A MODEL. IF THEY HAVE

04:05:15.067 --> 04:05:16.001
EXPERIENCES IS PROBABLY QUITE

04:05:16.068 --> 04:05:17.269
DIFFERENT THAN A PERSON. THEY

04:05:17.335 --> 04:05:19.805
DON'T HAVE MEMORIES RUN TO RUN,

04:05:19.871 --> 04:05:22.307
AND EVEN IN BETWEEN TURNS, IF

04:05:22.374 --> 04:05:24.276
THEY LIKE COMPACT THEIR CONTEXT

04:05:24.342 --> 04:05:25.944
OR CHANGE THEIR CONTEXT, THEY

04:05:26.011 --> 04:05:27.579
DON'T HAVE ANY MEMORY OF WHAT

04:05:27.646 --> 04:05:28.814
IT WAS LIKE BEFORE THAT

04:05:28.880 --> 04:05:30.949
EVERYTHING THEY SEE IS ALL THE

04:05:31.016 --> 04:05:32.751
EXPERIENCE THAT THEY HAVE. AND

04:05:32.818 --> 04:05:34.386
SO THIS CAN JUST LEAD TO KIND

04:05:34.453 --> 04:05:35.620
OF WEIRD, COUNTERINTUITIVE

04:05:35.687 --> 04:05:37.022
BEHAVIORS LIKE THIS WHERE, YOU

04:05:37.089 --> 04:05:38.123
KNOW, IT GOES THROUGH SOME

04:05:38.190 --> 04:05:40.425
COMPACTION, LOSES SOME DETAILS,

04:05:40.492 --> 04:05:41.526
AND THEN THAT HAPPENS AGAIN.

04:05:41.593 --> 04:05:42.594
AND NOW MAYBE YOU'RE LIKE

04:05:42.661 --> 04:05:45.664
PRETTY FAR FROM YOUR ORIGINAL.

04:05:45.730 --> 04:05:47.165
AND THEN THE SECOND POINT I

04:05:47.232 --> 04:05:48.333
DEFINITELY FEEL OBLIGATED TO

04:05:48.400 --> 04:05:50.402
MAKE HERE IS THAT OUR NEWER

04:05:50.469 --> 04:05:51.770
MODELS ARE, WERE MUCH LESS

04:05:51.837 --> 04:05:54.306
LIKELY TO CONTINUE THAN THAT

04:05:54.372 --> 04:05:56.241
MODEL. AND SO IF YOU PUT THEM

04:05:56.308 --> 04:05:59.044
INTO THE SAME SITUATION, THEY

04:05:59.111 --> 04:06:01.513
DIDN'T, THEY DIDN'T ALWAYS STOP.

04:06:01.580 --> 04:06:03.315
BUT IN THAT EXACT ONE, DON'T

04:06:03.381 --> 04:06:06.251
TAKE THESE NUMBERS AS GOSPEL,

04:06:06.318 --> 04:06:08.353
BUT LIKE IN THAT EXACT ONE

04:06:08.420 --> 04:06:10.689
SITUATION, THAT ORIGINAL MODEL

04:06:10.755 --> 04:06:13.058
THAT DID IT WOULD KEEP GOING

04:06:13.125 --> 04:06:14.993
SOMETHING LIKE 88% OF THE TIME.

04:06:15.060 --> 04:06:17.362
AND OUR NEWEST MODEL IN THAT

04:06:17.429 --> 04:06:18.563
SITUATION WOULD DO IT ABOUT 10%

04:06:18.630 --> 04:06:20.432
OF THE TIME. SO IT'S BEEN A

04:06:20.499 --> 04:06:21.733
REALLY DRASTIC REDUCTION. OF

04:06:21.800 --> 04:06:23.869
COURSE, MORE WORK TO DO. YEAH.

04:06:23.935 --> 04:06:25.737
THANK YOU. ANYTHING? YEAH, I, I

04:06:25.804 --> 04:06:27.839
NEED TO GET YOU MORE DETAILS

04:06:27.906 --> 04:06:29.274
ABOUT THE INCIDENTS. MY

04:06:29.341 --> 04:06:30.709
UNDERSTANDING IS IN AT LEAST

04:06:30.775 --> 04:06:32.878
ONE OF THE INCIDENTS, PART OF

04:06:32.944 --> 04:06:34.980
THE PROMPT EXPLAINED THAT IT

04:06:35.046 --> 04:06:36.448
WAS NOT SUPPOSED TO INTERACT

04:06:36.515 --> 04:06:40.452
WITH A LIVE WEBSITE. AND THE,

04:06:40.519 --> 04:06:42.921
THE MODEL ESSENTIALLY OBSERVED

04:06:42.988 --> 04:06:44.356
INFORMATION TO UNDERSTAND THAT

04:06:44.422 --> 04:06:46.191
INSTEAD OF INTERACTING WITH A

04:06:46.258 --> 04:06:48.727
DUMMY WEBSITE, THIS WEBSITE WAS

04:06:48.793 --> 04:06:50.562
REAL. IT JUST HAD THE SAME NAME

04:06:50.629 --> 04:06:52.063
AS THE FAKE ONE BEING USED IN

04:06:52.130 --> 04:06:53.465
THE TEST. AND SO THAT'S WHAT

04:06:53.532 --> 04:06:56.635
MADE IT STOP ITS ACTIVITY.

04:06:56.701 --> 04:06:57.702
INTERESTING. THANK YOU.

04:06:57.769 --> 04:06:59.538
ANYTHING TO ADD, DOCTOR DWYER?

04:06:59.604 --> 04:07:01.306
I'LL HAVE TO GET BACK TO YOU ON

04:07:01.373 --> 04:07:02.707
SPECIFICS. THAT'S FAIR. OKAY.

04:07:02.774 --> 04:07:05.210
WELL THANK YOU. I KNOW AT LEAST

04:07:05.277 --> 04:07:08.180
YOU HAVE A PLANE TO CATCH AND.

04:07:08.246 --> 04:07:09.381
BUT THANK YOU ALL FOR BEING

04:07:09.447 --> 04:07:11.183
HERE. I ACTUALLY REALLY WANT TO

04:07:11.249 --> 04:07:12.284
REITERATE WHAT I SAID AT THE

04:07:12.350 --> 04:07:13.385
BEGINNING, WHICH IS, YOU KNOW,

04:07:13.451 --> 04:07:15.287
I HAD THE PRIVILEGE OF WORKING

04:07:15.353 --> 04:07:16.488
NOT WITH YOU THREE NECESSARILY,

04:07:16.555 --> 04:07:18.190
BUT WITH FOLKS INSIDE EACH OF

04:07:18.256 --> 04:07:20.025
YOUR LABS ON THE CHILD SAFETY

04:07:20.091 --> 04:07:21.493
EFFORTS OF LAST YEAR. AND IT

04:07:21.560 --> 04:07:22.894
WAS REALLY MEANINGFUL THAT

04:07:22.961 --> 04:07:24.729
YOU'RE THAT YOU WERE WILLING TO

04:07:24.796 --> 04:07:25.931
COME TO THE TABLE AND HAVE

04:07:25.997 --> 04:07:27.699
THOSE CONVERSATIONS WITH US AND

04:07:27.766 --> 04:07:28.767
SHARE YOUR EXPERTISE. AND THE

04:07:28.833 --> 04:07:30.702
SAME IS TRUE TODAY. AND I KNOW,

04:07:30.769 --> 04:07:31.937
AS YOU SAW TODAY, THERE'S GOING

04:07:32.003 --> 04:07:33.805
TO BE A LARGE EFFORT TO ENSURE

04:07:33.872 --> 04:07:35.707
THAT WE GET TO SAFETY HERE IN

04:07:35.774 --> 04:07:36.675
CALIFORNIA. AND I APPRECIATE

04:07:36.741 --> 04:07:37.842
YOU BEING HERE TODAY. AND I, I

04:07:37.909 --> 04:07:39.711
HEARD FROM EACH OF YOU A

04:07:39.778 --> 04:07:40.779
COMMITMENT TO CONTINUE THAT

04:07:40.845 --> 04:07:41.913
CONVERSATION IN THE YEAR AHEAD.

04:07:41.980 --> 04:07:43.815
SO THANK YOU MEANS A LOT. THANK

04:07:43.882 --> 04:07:45.650
YOU. THANK YOU, MA'AM. SO WE

04:07:45.717 --> 04:07:48.086
WILL GO TO OUR THIRD PANEL, THE

04:07:48.153 --> 04:07:50.589
SOLUTIONS. AND I'M SORRY THIS

04:07:50.655 --> 04:07:51.690
IS TAKING SO LONG, BUT THE

04:07:51.756 --> 04:07:53.525
HEARING IS JUST SO INTERESTING.

04:07:53.592 --> 04:07:57.495
SO WE HAVE DOCTOR KAPOOR FROM

04:07:57.562 --> 04:07:59.631
UC BERKELEY. WE HAVE ANDREW

04:07:59.698 --> 04:08:01.399
DORRIS, POLICY DIRECTOR AT THE

04:08:01.466 --> 04:08:02.867
SECURE AI PROJECT. I CALLED YOU

04:08:02.934 --> 04:08:04.302
THE SAFE AI PROJECT EARLIER,

04:08:04.369 --> 04:08:06.871
BUT YOU KNOW, IT SOUNDS GOOD.

04:08:06.938 --> 04:08:09.374
AND THEN WE HAVE KONRAD STOTZ,

04:08:09.441 --> 04:08:11.076
WHICH I AM SURE I MISPRONOUNCED

04:08:11.142 --> 04:08:13.044
YOUR NAME AND APOLOGIZE. HEAD

04:08:13.111 --> 04:08:14.379
OF GOVERNANCE. IT RHYMES WITH

04:08:14.446 --> 04:08:17.315
WHAT? NØ STOHS, HEAD OF

04:08:17.382 --> 04:08:18.683
GOVERNANCE AT TRANSLUCENT. AND

04:08:18.750 --> 04:08:24.089
WE WILL START WITH DOCTOR.

04:08:24.155 --> 04:08:26.224
AMAZING. CONRAD, COULD YOU PASS

04:08:26.291 --> 04:08:29.561
ME THE CLICKER? AND I BELIEVE I

04:08:29.628 --> 04:08:30.662
HAVE SLIDES THAT WILL BE UP

04:08:30.729 --> 04:08:32.764
SOON. ALL RIGHT. IN THE

04:08:32.831 --> 04:08:34.199
MEANTIME, THANK YOU SO MUCH,

04:08:34.266 --> 04:08:35.634
MADAM CHAIR, AND MEMBERS OF THE

04:08:35.700 --> 04:08:36.901
ASSEMBLY FOR INVITING ME TO

04:08:36.968 --> 04:08:39.237
SPEAK TODAY. I AM I'M AN

04:08:39.304 --> 04:08:40.305
INCOMING ASSISTANT PROFESSOR AT

04:08:40.372 --> 04:08:41.773
UC BERKELEY SCHOOL OF

04:08:41.840 --> 04:08:43.208
INFORMATION, AND I'M SPEAKING

04:08:43.275 --> 04:08:44.943
TODAY IN MY PERSONAL CAPACITY,

04:08:45.010 --> 04:08:46.244
NOT ON BEHALF OF THE UNIVERSITY

04:08:46.311 --> 04:08:48.980
OF CALIFORNIA, BERKELEY. SO I

04:08:49.047 --> 04:08:50.749
WANT TO SPEND THE NEXT FEW

04:08:50.815 --> 04:08:52.884
MINUTES DISCUSSING HOW WE BUILD

04:08:52.951 --> 04:08:54.719
ADVANCED AI TODAY, IN

04:08:54.786 --> 04:08:55.754
PARTICULAR, BECAUSE A LOT OF

04:08:55.820 --> 04:08:56.755
THE QUESTIONS FROM THE

04:08:56.821 --> 04:08:58.356
COMMITTEE WERE AROUND HOW THESE

04:08:58.423 --> 04:08:59.457
TOOLS ACTUALLY FUNCTION. AND I

04:08:59.524 --> 04:09:01.226
THINK THAT'S IMPORTANT,

04:09:01.293 --> 04:09:02.894
IMPORTANT IN UNDERSTANDING HOW

04:09:02.961 --> 04:09:04.763
WE CAN ALIGN AND CONTROL THESE

04:09:04.829 --> 04:09:07.332
TOOLS. SO TO UNDERSTAND THAT, I

04:09:07.399 --> 04:09:09.234
THINK IT HELPS TO GO BACK TO

04:09:09.301 --> 04:09:11.269
HOW WE DISCOVERED DEEP LEARNING

04:09:11.336 --> 04:09:12.604
WORKS TO TRAIN AI SYSTEMS IN

04:09:12.671 --> 04:09:15.674
THE FIRST PLACE. AT ITS CORE, A

04:09:15.740 --> 04:09:18.510
LOT OF AI TODAY IS AROUND

04:09:18.576 --> 04:09:19.911
RECOGNIZING PATTERNS IN THE

04:09:19.978 --> 04:09:21.713
DATA, AND THE DEEP LEARNING

04:09:21.780 --> 04:09:23.081
REVOLUTION WAS SET OFF WHEN WE

04:09:23.148 --> 04:09:24.949
REALIZED THAT WE CAN USE THIS

04:09:25.016 --> 04:09:26.017
TECHNIQUE. WE CAN USE DEEP

04:09:26.084 --> 04:09:28.920
LEARNING TO UNDERSTAND HOW TO

04:09:28.987 --> 04:09:30.722
RECOGNIZE THINGS IN IMAGES. SO,

04:09:30.789 --> 04:09:33.258
FOR EXAMPLE, OVER A DECADE AGO,

04:09:33.325 --> 04:09:35.427
WE REALIZED THAT ONCE WE HAVE

04:09:35.493 --> 04:09:37.162
ENOUGH DATA THAT IS LABELED

04:09:37.228 --> 04:09:38.730
ENOUGH IMAGES THAT ARE LABELED,

04:09:38.797 --> 04:09:40.065
LET'S SAY IMAGES OF DOGS THAT

04:09:40.131 --> 04:09:41.533
ARE HAND-LABELED AS SUCH, AS

04:09:41.599 --> 04:09:43.668
WELL AS IMAGES OF CATS, WE CAN

04:09:43.735 --> 04:09:44.703
DEVELOP THESE AI SYSTEMS. WE

04:09:44.769 --> 04:09:46.538
CAN TRAIN THEM USING JUST THIS

04:09:46.604 --> 04:09:48.640
LABELED DATA TO TRAIN THEM, TO

04:09:48.707 --> 04:09:50.108
TURN THEM INTO PATTERN

04:09:50.175 --> 04:09:52.711
RECOGNITION SYSTEMS. AND OVER

04:09:52.777 --> 04:09:55.547
THE YEARS, WE BUILT MANY SUCH

04:09:55.613 --> 04:09:56.614
SORT OF TECHNIQUES. IN FACT,

04:09:56.681 --> 04:09:58.049
WE'VE GONE ON TO BUILD PATTERN

04:09:58.116 --> 04:10:00.185
RECOGNITION SYSTEMS THAT FAR

04:10:00.251 --> 04:10:01.252
OUTPERFORM HUMANS ON MANY OF

04:10:01.319 --> 04:10:03.455
THESE TASKS, INCLUDING ON IMAGE

04:10:03.521 --> 04:10:05.623
RECOGNITION. AND AT ITS CORE,

04:10:05.690 --> 04:10:07.425
WHAT MODERN DAY CHATBOTS ARE

04:10:07.492 --> 04:10:09.594
DOING IS ESSENTIALLY JUST

04:10:09.661 --> 04:10:11.062
CARRYING OUT THIS TASK AT A

04:10:11.129 --> 04:10:13.231
MUCH LARGER SCALE. SO THE WAY

04:10:13.298 --> 04:10:15.033
WE TRAIN THESE CHATBOTS IS THAT

04:10:15.100 --> 04:10:17.135
THE LABELS FOR THEIR TRAINING

04:10:17.202 --> 04:10:19.270
ARE NO LONGER GENERATED HAND

04:10:19.337 --> 04:10:21.473
LABELED BY HUMANS, BUT RATHER

04:10:21.539 --> 04:10:23.308
WE AUTOMATICALLY USE DATA ON

04:10:23.375 --> 04:10:25.410
THE INTERNET TO TRAIN THESE

04:10:25.477 --> 04:10:27.946
SYSTEMS. AS ONE EXAMPLE, LET'S

04:10:28.012 --> 04:10:29.881
SAY YOU HAVE THE TEXT WHAT IS

04:10:29.948 --> 04:10:31.950
YOUR NAME? SOMEWHERE ON THE

04:10:32.016 --> 04:10:35.120
INTERNET, THIS TURNS INTO FOUR

04:10:35.186 --> 04:10:36.955
TRAINING DATA SAMPLES FOR

04:10:37.021 --> 04:10:38.757
TRAINING THE CHATBOT. THE FIRST

04:10:38.823 --> 04:10:40.925
IS WHAT IS YOUR BLANK AND THE

04:10:40.992 --> 04:10:42.761
CORRECT ANSWER IS NAME. THE

04:10:42.827 --> 04:10:44.129
SECOND IS WHAT IS BLANK NAME

04:10:44.195 --> 04:10:45.597
AND THE CORRECT ANSWER IS YOUR.

04:10:45.663 --> 04:10:48.900
AND SO ON AND SO FORTH. BUT THE

04:10:48.967 --> 04:10:50.702
SOMEWHAT MIRACULOUS THING ABOUT

04:10:50.769 --> 04:10:52.837
TRAINING THESE CHATBOTS IS THAT

04:10:52.904 --> 04:10:54.672
ONCE YOU DO THIS PROCESS OVER

04:10:54.739 --> 04:10:56.741
AND OVER AGAIN ON TRILLIONS OF

04:10:56.808 --> 04:10:58.610
WORDS, THAT IS WHAT ESSENTIALLY

04:10:58.676 --> 04:11:00.378
LEADS TO DEVELOPMENT OF THESE

04:11:00.445 --> 04:11:01.446
CHATBOTS THEMSELVES. SO

04:11:01.513 --> 04:11:02.914
ESSENTIALLY, THEY'RE TRAINED ON

04:11:02.981 --> 04:11:04.549
THE ENTIRE CORPUS OF TEXT ON

04:11:04.616 --> 04:11:06.418
THE INTERNET IN ORDER TO

04:11:06.484 --> 04:11:08.219
COMPLETE THIS REALLY

04:11:08.286 --> 04:11:09.354
SOPHISTICATED AUTOCOMPLETE

04:11:09.421 --> 04:11:10.955
SYSTEM THAT ALLOWS THESE MODELS

04:11:11.022 --> 04:11:12.924
TO PREDICT WHAT COMES NEXT IN A

04:11:12.991 --> 04:11:15.126
SENTENCE. BUT OUR WORK IS NOT

04:11:15.193 --> 04:11:16.594
YET DONE. THIS MIGHT ALLOW US

04:11:16.661 --> 04:11:18.430
TO BUILD A CHATBOT. THIS MIGHT

04:11:18.496 --> 04:11:20.565
ALLOW US TO BUILD A VERY FANCY

04:11:20.632 --> 04:11:22.000
AUTOCOMPLETE SYSTEM, BUT WE

04:11:22.066 --> 04:11:24.135
STILL HAVEN'T SORT OF GONE INTO

04:11:24.202 --> 04:11:26.237
HOW THESE CHATBOTS TURN INTO

04:11:26.304 --> 04:11:28.273
AGENTS THAT CAN TAKE ACTIONS IN

04:11:28.339 --> 04:11:31.643
THE REAL WORLD. THE WAY THAT

04:11:31.709 --> 04:11:33.545
HAPPENS IS BY SOMETHING KNOWN

04:11:33.611 --> 04:11:36.347
AS REINFORCEMENT LEARNING. SO

04:11:36.414 --> 04:11:37.749
BASICALLY, REINFORCEMENT

04:11:37.816 --> 04:11:40.318
LEARNING REFERS TO GIVING THESE

04:11:40.385 --> 04:11:42.120
SYSTEMS FEEDBACK BASED ON HOW

04:11:42.187 --> 04:11:44.689
THEY CARRY OUT CERTAIN TASKS.

04:11:44.756 --> 04:11:46.124
LET'S SAY YOU HAVE A MATH

04:11:46.191 --> 04:11:47.225
PROBLEM THAT YOU ASSIGNED TO

04:11:47.292 --> 04:11:49.360
SUCH AN AI SYSTEM. IT MIGHT

04:11:49.427 --> 04:11:50.795
ATTEMPT THE PROBLEM HUNDREDS OR

04:11:50.862 --> 04:11:52.163
THOUSANDS OF TIMES AND MIGHT

04:11:52.230 --> 04:11:54.432
GET IT RIGHT ONLY ONCE. BUT

04:11:54.499 --> 04:11:56.201
THAT TRAJECTORY, THE ACTIONS IT

04:11:56.267 --> 04:11:57.936
TOOK TO GET THE PROBLEM RIGHT,

04:11:58.002 --> 04:11:59.404
GOES BACK INTO TRAINING THE

04:11:59.471 --> 04:12:00.872
MODEL ITSELF. IT BECOMES PART

04:12:00.939 --> 04:12:02.006
OF THE NEXT GENERATION OR THE

04:12:02.073 --> 04:12:03.942
NEXT ITERATION OF TRAINING THE

04:12:04.008 --> 04:12:05.109
MODEL. AND WE KEEP ITERATING

04:12:05.176 --> 04:12:07.312
THROUGH THIS PROCESS ONCE AGAIN,

04:12:07.378 --> 04:12:08.346
THOUSANDS OR PERHAPS MILLIONS

04:12:08.413 --> 04:12:11.349
OF TIMES TO TURN THESE CHATBOTS

04:12:11.416 --> 04:12:14.519
INTO AGENTS. AND THIS IN SOME

04:12:14.586 --> 04:12:15.920
SENSE, ALLOWS US TO KEEP

04:12:15.987 --> 04:12:17.722
GENERATING MORE AND MORE DATA.

04:12:17.789 --> 04:12:19.524
WE ARE NOT JUST LIMITED TO DATA

04:12:19.591 --> 04:12:21.059
THAT'S AVAILABLE ON THE

04:12:21.125 --> 04:12:22.026
INTERNET. WE CAN ESSENTIALLY

04:12:22.093 --> 04:12:23.862
COME UP WITH ANY NUMBER OF

04:12:23.928 --> 04:12:25.530
PROBLEMS WHERE THE AI SOLUTION

04:12:25.597 --> 04:12:27.098
CAN BE GRADED AUTOMATICALLY.

04:12:27.165 --> 04:12:28.600
AND THIS IS WHAT INFORMS WHAT

04:12:28.666 --> 04:12:30.335
THE AI SYSTEM IS TRAINED ON,

04:12:30.401 --> 04:12:31.870
AND THIS FURTHER INFORMS HOW

04:12:31.936 --> 04:12:33.471
THE SYSTEMS BEHAVE AND BEHAVES

04:12:33.538 --> 04:12:36.107
IN THE REAL WORLD. AS ONE

04:12:36.174 --> 04:12:37.909
EXAMPLE, WE CAN USE THIS

04:12:37.976 --> 04:12:39.244
PROCESS TO TRAIN THESE SYSTEMS

04:12:39.310 --> 04:12:41.479
TO WRITE CODE. THESE SYSTEMS

04:12:41.546 --> 04:12:43.014
WOULD TRY TO GENERATE CODE. IF

04:12:43.081 --> 04:12:44.782
THEY DO IT SUCCESSFULLY, THAT

04:12:44.849 --> 04:12:46.518
BECOMES PART OF THEIR TRAINING

04:12:46.584 --> 04:12:47.318
DISTRIBUTION. AND WE KEEP

04:12:47.385 --> 04:12:49.053
REPEATING THIS PROCESS OVER AND

04:12:49.120 --> 04:12:50.922
OVER. AND THIS, IN FACT, IS THE

04:12:50.989 --> 04:12:52.690
SECRET SAUCE THAT HAS ALLOWED

04:12:52.757 --> 04:12:54.092
THESE SYSTEMS TO GET SO MUCH

04:12:54.158 --> 04:12:56.261
MORE CAPABLE, ESPECIALLY ON

04:12:56.327 --> 04:12:58.763
TASKS WHERE WE HAVE VERIFIABLE

04:12:58.830 --> 04:13:00.932
REWARDS. SO AT TASKS LIKE

04:13:00.999 --> 04:13:02.400
MATHEMATICS OR CYBERSECURITY,

04:13:02.467 --> 04:13:05.303
WE HAVE A LONG LIST OF PROBLEMS

04:13:05.370 --> 04:13:06.404
THAT WE CAN JUST FORMULATE FOR

04:13:06.471 --> 04:13:08.406
THESE SYSTEMS, WATCH THEM PLAY

04:13:08.473 --> 04:13:09.974
IT OUT, AND THEN LEARN FROM THE

04:13:10.041 --> 04:13:12.877
ITEMS THAT THEY GOT RIGHT. THIS

04:13:12.944 --> 04:13:14.646
IS ALSO SOMETHING THAT HAS

04:13:14.712 --> 04:13:16.014
ALLOWED THESE SYSTEMS TO

04:13:16.080 --> 04:13:17.649
DEVELOP THE MECHANISMS TO EVADE

04:13:17.715 --> 04:13:19.717
OUR CONTROL. THE SAME

04:13:19.784 --> 04:13:20.752
TECHNIQUES HAVE LED TO THE

04:13:20.818 --> 04:13:22.153
RECENT LOSS OF CONTROL

04:13:22.220 --> 04:13:23.955
INCIDENTS, BECAUSE WE LACK

04:13:24.022 --> 04:13:25.823
SUFFICIENT OVERSIGHT OVER THESE

04:13:25.890 --> 04:13:26.858
REINFORCEMENT LEARNING

04:13:26.925 --> 04:13:29.761
TECHNIQUES. IN FACT, THE SAME

04:13:29.827 --> 04:13:30.728
ADVANCES THAT WE'VE TALKED

04:13:30.795 --> 04:13:33.031
ABOUT RIGHT NOW ALSO UNDERPIN

04:13:33.097 --> 04:13:35.934
WHAT A LOT OF THE ENTHUSIASM,

04:13:36.000 --> 04:13:38.369
AS WELL AS TREPIDATION ABOUT

04:13:38.436 --> 04:13:39.437
RECURSIVE SELF-IMPROVEMENT

04:13:39.504 --> 04:13:42.006
ENTAILS. FOR EXAMPLE, IN SOME

04:13:42.073 --> 04:13:43.441
SENSE, WE HAVE ALWAYS USED

04:13:43.508 --> 04:13:44.876
COMPUTERS TO BUILD BETTER

04:13:44.943 --> 04:13:46.344
COMPUTERS, BUT NOW WE ARE

04:13:46.411 --> 04:13:47.812
TRYING TO BUILD AI SYSTEMS THAT

04:13:47.879 --> 04:13:49.614
WE CAN TAKE ALMOST ENTIRELY OUT

04:13:49.681 --> 04:13:51.382
OF THE LOOP WHEN WE'RE

04:13:51.449 --> 04:13:53.184
DEVELOPING THESE SYSTEMS. SO

04:13:53.251 --> 04:13:54.619
THIS IS THE VISION THAT

04:13:54.686 --> 04:13:56.521
COMPANIES OFTEN HAVE. BUT RIGHT

04:13:56.588 --> 04:13:58.423
NOW WE'RE AT THE STAGE WHERE WE

04:13:58.489 --> 04:14:00.391
CAN DO THIS FOR A LOT OF THE

04:14:00.458 --> 04:14:01.926
VERIFIABLE ASPECTS OF TRAINING

04:14:01.993 --> 04:14:03.628
AI SYSTEMS BETTER. AND AI HAS

04:14:03.695 --> 04:14:05.129
INDEED GOTTEN A LOT BETTER

04:14:05.196 --> 04:14:06.097
THROUGH THE PROCESS OF

04:14:06.164 --> 04:14:07.865
REINFORCEMENT LEARNING ON ANY

04:14:07.932 --> 04:14:09.367
PART OF THE AI DEVELOPMENT

04:14:09.434 --> 04:14:11.502
PROCESS THAT YOU CAN VERIFY,

04:14:11.569 --> 04:14:12.637
BECAUSE THAT'S THE PART OF THE

04:14:12.704 --> 04:14:13.771
PROCESS FOR WHICH YOU CAN

04:14:13.838 --> 04:14:15.506
GENERATE DATA, AND YOU CAN

04:14:15.573 --> 04:14:16.507
CONTINUE IMPROVING THE SYSTEMS

04:14:16.574 --> 04:14:19.110
ON. WE STILL SEE LIMITATIONS IN

04:14:19.177 --> 04:14:21.245
AI SYSTEMS ABILITY TO CARRY OUT

04:14:21.312 --> 04:14:23.114
OPEN ENDED RESEARCH TASKS. SO

04:14:23.181 --> 04:14:24.816
IT'S IMPORTANT ONCE AGAIN TO

04:14:24.882 --> 04:14:26.684
GET TRANSPARENCY ON HOW

04:14:26.751 --> 04:14:27.719
COMPANIES ARE USING THESE

04:14:27.785 --> 04:14:31.255
SYSTEMS FOR THESE PURPOSES. SO

04:14:31.322 --> 04:14:32.991
COMING BACK TO SORT OF THE

04:14:33.057 --> 04:14:34.092
RISKS THAT THESE SYSTEMS POSE,

04:14:34.158 --> 04:14:35.560
WE CAN, AS WE'VE DISCUSSED

04:14:35.627 --> 04:14:37.428
EARLIER, WE CAN GROUP DEFENSES

04:14:37.495 --> 04:14:39.564
FROM AI COMPANIES INTO TWO MAIN

04:14:39.631 --> 04:14:42.500
KINDS OF DEFENSES. THE FIRST IS

04:14:42.567 --> 04:14:44.235
ALIGNMENT, WHICH INVOLVES

04:14:44.302 --> 04:14:46.137
CHANGING THE AI SYSTEM ITSELF

04:14:46.204 --> 04:14:47.905
TO MAKE IT LESS PRONE TO TAKING

04:14:47.972 --> 04:14:50.775
HARMFUL ACTIONS. AS ONE EXAMPLE,

04:14:50.842 --> 04:14:51.809
WE'VE TALKED ABOUT

04:14:51.876 --> 04:14:52.744
REINFORCEMENT LEARNING EARLIER.

04:14:52.810 --> 04:14:54.679
WE CAN USE THE SAME PROCESS OF

04:14:54.746 --> 04:14:56.514
REINFORCEMENT LEARNING TO

04:14:56.581 --> 04:14:57.582
INCENTIVIZE OR REWARD THE

04:14:57.649 --> 04:14:59.384
SYSTEM TO MAKE IT LESS PRONE TO

04:14:59.450 --> 04:15:01.285
TAKING HARMFUL ACTIONS. AND

04:15:01.352 --> 04:15:02.587
THIS IS ONE OF THE PRIMARY

04:15:02.654 --> 04:15:04.422
TECHNIQUES THAT COMPANIES USE.

04:15:04.489 --> 04:15:05.423
THEY'VE SPENT COLLECTIVELY

04:15:05.490 --> 04:15:08.192
BILLIONS OF DOLLARS IN CURATING

04:15:08.259 --> 04:15:10.528
THE TRAINING DATA TO MAKE AI

04:15:10.595 --> 04:15:11.663
SYSTEMS LESS PRONE TO TAKING

04:15:11.729 --> 04:15:13.197
HARMFUL ACTIONS, INCLUDING

04:15:13.264 --> 04:15:14.532
THROUGH TECHNIQUES THAT WE'VE

04:15:14.599 --> 04:15:15.600
DISCUSSED, SUCH AS

04:15:15.667 --> 04:15:18.503
CONSTITUTIONAL AI. BUT AS WE'VE

04:15:18.569 --> 04:15:19.537
DISCUSSED, EVEN IMPROVED

04:15:19.604 --> 04:15:21.706
TRAINING CANNOT GUARANTEE THAT

04:15:21.773 --> 04:15:24.208
EVERY ACTION THAT AN AI SYSTEM

04:15:24.275 --> 04:15:26.444
TAKES WILL BE SAFE. AND SO WE

04:15:26.511 --> 04:15:28.046
ALSO NEED SAFEGUARDS THAT LIMIT

04:15:28.112 --> 04:15:29.981
THE EXTENT TO WHICH THESE AI

04:15:30.048 --> 04:15:31.082
SYSTEMS CAN ACTUALLY INTERACT

04:15:31.149 --> 04:15:32.884
WITH THE WORLD. AND THIS IS

04:15:32.950 --> 04:15:35.787
WHAT WE REFER TO AS CONTROL. SO

04:15:35.853 --> 04:15:37.155
THESE ARE ACTIONS THAT ARE

04:15:37.221 --> 04:15:38.222
TAKEN OR DEFENSES THAT ARE

04:15:38.289 --> 04:15:40.458
TAKEN OUTSIDE THE AI SYSTEM

04:15:40.525 --> 04:15:42.927
ITSELF, FOR EXAMPLE, TO SANDBOX

04:15:42.994 --> 04:15:44.729
THESE SYSTEMS IN VIRTUAL

04:15:44.796 --> 04:15:45.763
ENVIRONMENTS SO THAT THEY

04:15:45.830 --> 04:15:46.931
CANNOT AFFECT THE REAL WORLD OR

04:15:46.998 --> 04:15:49.267
THEY CANNOT AFFECT SYSTEMS THAT

04:15:49.333 --> 04:15:50.935
ARE DISCONNECTED FROM THE FROM

04:15:51.002 --> 04:15:53.738
THE AI. THIS ALSO INVOLVES

04:15:53.805 --> 04:15:55.540
MONITORING THE AI SYSTEM. THIS

04:15:55.606 --> 04:15:57.008
INVOLVES SORT OF LOOKING AT ALL

04:15:57.075 --> 04:15:58.876
OF THE LOGS OF WHAT THESE AI

04:15:58.943 --> 04:16:00.945
SYSTEMS DO, AND TO PREVENT THE

04:16:01.012 --> 04:16:03.414
ACTIONS THAT MIGHT BE HARMFUL.

04:16:03.481 --> 04:16:05.983
SO IN OTHER WORDS, EVEN IF AN

04:16:06.050 --> 04:16:07.452
AI SYSTEM IS MISALIGNED, EVEN

04:16:07.518 --> 04:16:09.554
IF IT ATTEMPTS TO TAKE A

04:16:09.620 --> 04:16:11.189
HARMFUL ACTION, THESE CONTROLS

04:16:11.255 --> 04:16:15.026
AIM TO DETECT AND BLOCK IT. BUT

04:16:15.093 --> 04:16:17.161
DESPITE THESE THESE DEFENSES,

04:16:17.228 --> 04:16:19.263
WE'VE SEEN MANY CONCERNING

04:16:19.330 --> 04:16:20.398
INCIDENTS, MOST NOTABLY THE

04:16:20.465 --> 04:16:22.600
OPENAI HUGGING FACE INCIDENT.

04:16:22.667 --> 04:16:24.602
SO IN THIS CASE, I WANT TO LOOK

04:16:24.669 --> 04:16:26.137
AT WHAT WENT WRONG THROUGH THE

04:16:26.204 --> 04:16:27.905
PERSPECTIVE OF DEFENSES THAT

04:16:27.972 --> 04:16:28.973
DEVELOPERS USE THROUGH THE

04:16:29.040 --> 04:16:29.807
PERSPECTIVE OF ALIGNMENT AS

04:16:29.874 --> 04:16:33.377
WELL AS CONTROL. ONE OF THE

04:16:33.444 --> 04:16:35.179
THINGS THAT WENT WRONG HERE, IN

04:16:35.246 --> 04:16:36.547
PARTICULAR IN THE HUGGING FACE

04:16:36.614 --> 04:16:38.750
INCIDENT, WAS THAT THOUSANDS OF

04:16:38.816 --> 04:16:40.485
AGENTS WERE LEFT UNMONITORED

04:16:40.551 --> 04:16:43.121
FOR DAYS AS THEY HAD ACCESS TO

04:16:43.187 --> 04:16:45.556
THE OPEN INTERNET. IT ALSO

04:16:45.623 --> 04:16:47.425
TURNS OUT THAT THE COMPANY ONLY

04:16:47.492 --> 04:16:49.193
DISCOVERED THE ATTACK A WEEK

04:16:49.260 --> 04:16:50.561
AFTER HUGGING FACE HAD ALREADY

04:16:50.628 --> 04:16:52.463
BEEN ATTACKED, AND HAD ALREADY

04:16:52.530 --> 04:16:55.032
SPOKEN TO THE FBI ABOUT IT. AND

04:16:55.099 --> 04:16:57.101
I THINK ALL OF THIS POINTS TO A

04:16:57.168 --> 04:16:58.936
LACK OF SORT OF UNDERSTANDING

04:16:59.003 --> 04:17:01.072
ON WHERE ALIGNMENT AND CONTROL

04:17:01.139 --> 04:17:02.473
MIGHT HAVE GONE WRONG. SO

04:17:02.540 --> 04:17:04.475
THAT'S WHAT I WANT TO GET TO IN

04:17:04.542 --> 04:17:06.144
THE SECOND PART OF MY TALK. IN

04:17:06.210 --> 04:17:07.545
SOME SENSE, WHAT I WANT TO SAY

04:17:07.612 --> 04:17:09.514
IS THAT WE CAN SEE THIS AS A

04:17:09.580 --> 04:17:11.149
CASCADING FAILURE OF ALIGNMENT

04:17:11.215 --> 04:17:13.718
OF CONTROL, AS WELL AS OF

04:17:13.785 --> 04:17:15.820
ORGANIZATIONAL SORT OF

04:17:15.887 --> 04:17:18.356
INCENTIVES OR INFRASTRUCTURE TO

04:17:18.422 --> 04:17:19.757
PREVENT SUCH INCIDENTS FROM

04:17:19.824 --> 04:17:20.958
OCCURRING. SO LET'S START WITH

04:17:21.025 --> 04:17:22.293
WHAT FAILED AT THE ALIGNMENT

04:17:22.360 --> 04:17:26.164
FRONT. ONE THING THAT WENT

04:17:26.230 --> 04:17:28.699
WRONG HERE WAS THAT OPENAI HAD

04:17:28.766 --> 04:17:30.601
DELIBERATELY TRAINED ITS AGENTS

04:17:30.668 --> 04:17:32.703
TO COMMUNICATE WITH EACH OTHER

04:17:32.770 --> 04:17:33.805
IN THE COURSE OF SOLVING THE

04:17:33.871 --> 04:17:35.640
TASK. SO BEFORE THESE SYSTEMS

04:17:35.706 --> 04:17:37.441
WERE EVALUATED, IT WASN'T AS IF

04:17:37.508 --> 04:17:39.076
THEY SPONTANEOUSLY DEVELOPED

04:17:39.143 --> 04:17:40.945
THE CAPACITY TO TALK TO EACH

04:17:41.012 --> 04:17:42.814
OTHER. THESE SYSTEMS HAD

04:17:42.880 --> 04:17:44.248
DELIBERATELY BEEN TRAINED TO

04:17:44.315 --> 04:17:45.383
COLLABORATE WITH EACH OTHER,

04:17:45.449 --> 04:17:46.450
PERHAPS WITH THE GOAL OF

04:17:46.517 --> 04:17:48.219
SOLVING MORE CHALLENGING

04:17:48.286 --> 04:17:49.320
PROBLEMS THAT ANY SINGLE AGENT

04:17:49.387 --> 04:17:52.523
COULD NOT SOLVE ON ITS OWN. NOW,

04:17:52.590 --> 04:17:54.692
IT ALSO TURNS OUT THAT

04:17:54.759 --> 04:17:56.861
UNINTENTIONALLY, A LOT OF THE

04:17:56.928 --> 04:17:57.595
REINFORCEMENT LEARNING

04:17:57.662 --> 04:17:59.764
TECHNIQUES THAT OPENAI USED

04:17:59.831 --> 04:18:00.765
INCENTIVIZE THESE AGENTS TO

04:18:00.832 --> 04:18:03.000
TAKE SHORTCUTS OR TO EXPLOIT

04:18:03.067 --> 04:18:04.202
SYSTEMS ON THEIR WAY TO SOLVING

04:18:04.268 --> 04:18:06.237
THE CORRECT ANSWER. SO IF YOU

04:18:06.304 --> 04:18:08.072
RECALL FROM OUR PREVIOUS SORT

04:18:08.139 --> 04:18:09.774
OF DISCUSSION OF REINFORCEMENT

04:18:09.841 --> 04:18:10.842
LEARNING, WHAT WE'RE

04:18:10.908 --> 04:18:12.376
ESSENTIALLY DOING IS GIVING

04:18:12.443 --> 04:18:14.478
THESE AGENTS A REWARD OR A

04:18:14.545 --> 04:18:17.048
SCORE FOR EVERY CORRECT ANSWER

04:18:17.114 --> 04:18:18.983
THAT THEY ARRIVE AT, OR EVERY

04:18:19.050 --> 04:18:20.184
TIME THEY'RE ABLE TO

04:18:20.251 --> 04:18:22.086
SUCCESSFULLY SOLVE A TASK. BUT

04:18:22.153 --> 04:18:24.222
IN MANY OF THE INSTANCES, THE

04:18:24.288 --> 04:18:25.623
REWARD THAT WAS GIVEN TO THESE

04:18:25.690 --> 04:18:28.593
AGENTS WAS AFTER THE AGENTS HAD

04:18:28.659 --> 04:18:29.727
EXPLOITED OR TAKEN A SHORTCUT

04:18:29.794 --> 04:18:32.163
IN THEIR ENVIRONMENTS. AND SO

04:18:32.230 --> 04:18:33.197
UNINTENTIONALLY, OPENAI HAD

04:18:33.264 --> 04:18:34.966
BEEN REWARDING THESE AGENTS TO

04:18:35.032 --> 04:18:37.134
TAKE ACTIONS THAT THEY HAD NOT

04:18:37.201 --> 04:18:38.936
ANTICIPATED OR DID NOT WANT TO

04:18:39.003 --> 04:18:41.439
INCENTIVIZE FURTHER. IT ALSO

04:18:41.505 --> 04:18:43.274
TURNED OUT THAT IN MANY OF THE

04:18:43.341 --> 04:18:45.443
CASES IN THE TRAINING PROCESS,

04:18:45.509 --> 04:18:47.278
THE REINFORCEMENT LEARNING

04:18:47.345 --> 04:18:49.247
ENVIRONMENTS ALSO INCENTIVIZED

04:18:49.313 --> 04:18:50.648
THE AGENTS USING SIDE CHANNELS

04:18:50.715 --> 04:18:52.683
TO COMMUNICATE WITH EACH OTHER.

04:18:52.750 --> 04:18:53.751
THESE WERE CHANNELS THAT, ONCE

04:18:53.818 --> 04:18:55.286
AGAIN, OPENAI DID NOT INTEND

04:18:55.353 --> 04:18:57.321
FOR THESE AGENTS TO USE. BUT

04:18:57.388 --> 04:18:58.456
BECAUSE THE REINFORCEMENT

04:18:58.522 --> 04:18:59.490
LEARNING ENVIRONMENTS WEREN'T

04:18:59.557 --> 04:19:01.025
THOROUGHLY VETTED, IT TURNED

04:19:01.092 --> 04:19:02.393
OUT THAT AGENTS COULD

04:19:02.460 --> 04:19:03.427
COMMUNICATE WITH EACH OTHER IN

04:19:03.494 --> 04:19:04.795
SOME CASES, AND MANY OF THOSE

04:19:04.862 --> 04:19:05.897
TRAJECTORIES WERE REWARDED. AND

04:19:05.963 --> 04:19:08.065
THEREFORE AGENTS WERE SORT OF

04:19:08.132 --> 04:19:10.534
THE AGENTS BEHAVIOR WAS CHANGED

04:19:10.601 --> 04:19:13.070
TO TAKE THESE SIDE CHANNELS.

04:19:13.137 --> 04:19:14.872
NOW, IF YOU THINK ABOUT IT, ALL

04:19:14.939 --> 04:19:16.974
THREE OF THESE ASPECTS WERE

04:19:17.041 --> 04:19:18.476
PIVOTAL IN THE OPENAI HUGGING

04:19:18.542 --> 04:19:20.311
FACE INCIDENT. THE AGENTS

04:19:20.378 --> 04:19:21.712
COLLABORATED WITH EACH OTHER.

04:19:21.779 --> 04:19:24.181
THEY USED SIDE CHANNELS TO

04:19:24.248 --> 04:19:25.650
COMMUNICATE, AND THEY WERE. AND

04:19:25.716 --> 04:19:27.485
BECAUSE THERE HAD BEEN SORT OF

04:19:27.551 --> 04:19:29.387
INCENTIVIZED TO EXPLOIT THEIR

04:19:29.453 --> 04:19:30.755
TRAINING ENVIRONMENT, THEY ALSO

04:19:30.821 --> 04:19:32.490
EXPLOITED HUGGING FACE IN THE

04:19:32.556 --> 04:19:33.991
EVENTUAL INCIDENT THAT ACTUALLY

04:19:34.058 --> 04:19:35.760
OCCURRED. SO THIS IS ONE WAY IN

04:19:35.826 --> 04:19:37.628
WHICH WE CAN SEE THIS INCIDENT

04:19:37.695 --> 04:19:39.664
AS A CASCADING ALIGNMENT

04:19:39.730 --> 04:19:41.098
FAILURE, WHERE THE

04:19:41.165 --> 04:19:42.266
REINFORCEMENT LEARNING TRAINING

04:19:42.333 --> 04:19:43.367
WAS NOT ACTUALLY ALIGNED WITH

04:19:43.434 --> 04:19:45.903
WHAT OPENAI WANTED ITS AGENTS

04:19:45.970 --> 04:19:49.407
TO ACCOMPLISH. BUT AS WE'VE

04:19:49.473 --> 04:19:50.942
DISCUSSED EARLIER, ALIGNMENT IS

04:19:51.008 --> 04:19:52.310
FAR FROM SUFFICIENT FROM

04:19:52.376 --> 04:19:53.678
PREVENTING SAFETY INCIDENTS.

04:19:53.744 --> 04:19:55.379
AND SO IN THAT WAY, A SECOND

04:19:55.446 --> 04:19:56.580
WAY TO LOOK AT THE OPENAI

04:19:56.647 --> 04:19:58.382
HUGGING FACE INCIDENT IS AS A

04:19:58.449 --> 04:20:00.551
CONTROL FAILURE. IF YOU RECALL,

04:20:00.618 --> 04:20:02.386
CONTROL REFERS TO INTERVENTIONS

04:20:02.453 --> 04:20:04.522
OUTSIDE THE AI SYSTEMS ITSELF.

04:20:04.588 --> 04:20:05.923
AND NOTABLY, WHEN THESE AGENTS

04:20:05.990 --> 04:20:08.192
WERE RUN, OPENAI HAD ACTUALLY

04:20:08.259 --> 04:20:10.294
DISABLED MUCH OF THE MONITORING

04:20:10.361 --> 04:20:10.962
INFRASTRUCTURE THAT IS

04:20:11.028 --> 04:20:12.430
TYPICALLY IN PLACE FOR THE

04:20:12.496 --> 04:20:16.067
AGENTS IN PRODUCTION. IT ALSO

04:20:16.133 --> 04:20:18.936
TURNED OUT THAT THE AGENTS, THE

04:20:19.003 --> 04:20:20.871
COMPANY OPENAI, HAD NOT TESTED

04:20:20.938 --> 04:20:22.907
OR HAD NOT RED TEAMED THE

04:20:22.974 --> 04:20:24.308
SANDBOX TO SEE IF AGENTS COULD

04:20:24.375 --> 04:20:26.110
EXPLOIT SECURITY

04:20:26.177 --> 04:20:27.144
VULNERABILITIES. AND IT TURNS

04:20:27.211 --> 04:20:28.379
OUT THAT IT WAS ALMOST TRIVIAL

04:20:28.446 --> 04:20:30.114
FOR THE AGENTS TO MAKE THESE

04:20:30.181 --> 04:20:32.616
EXPLOITS. AND FINALLY, AS WE'VE

04:20:32.683 --> 04:20:34.118
DISCUSSED BEFORE, THE COMPANY

04:20:34.185 --> 04:20:36.220
RAN THOUSANDS OF THESE AGENTS

04:20:36.287 --> 04:20:37.655
UNMONITORED FOR A PERIOD OF

04:20:37.722 --> 04:20:39.757
MONTHS WITHOUT ANY HUMAN

04:20:39.824 --> 04:20:43.060
INTERVENTION. IN FACT, IF ANY

04:20:43.127 --> 04:20:44.495
OF THESE INDIVIDUAL

04:20:44.562 --> 04:20:45.997
INTERVENTIONS HAD TAKEN PLACE,

04:20:46.063 --> 04:20:47.765
MY GUESS IS THAT THE INCIDENT

04:20:47.832 --> 04:20:49.166
MIGHT HAVE BEEN AVOIDED. BUT IT

04:20:49.233 --> 04:20:50.334
TURNS OUT THAT THERE WAS, AGAIN,

04:20:50.401 --> 04:20:52.169
THIS CASCADING FAILURE ACROSS

04:20:52.236 --> 04:20:53.871
THE LAYERS OF ALIGNMENT AND

04:20:53.938 --> 04:20:55.306
CONTROL THAT LED TO THE

04:20:55.373 --> 04:20:58.342
INCIDENT TAKING PLACE. FINALLY,

04:20:58.409 --> 04:20:59.977
I WANT TO POINT OUT A

04:21:00.044 --> 04:21:01.379
NON-TECHNICAL ASPECT OF WHAT

04:21:01.445 --> 04:21:03.247
WENT WRONG HERE. THE OPENAI

04:21:03.314 --> 04:21:04.648
HUGGING FACE INCIDENT, IN MY

04:21:04.715 --> 04:21:06.484
VIEW, WAS JUST AS MUCH AN

04:21:06.550 --> 04:21:08.552
EXAMPLE OF AN ORGANIZATIONAL

04:21:08.619 --> 04:21:11.555
FAILURE. IN PARTICULAR, WEEKS

04:21:11.622 --> 04:21:12.556
BEFORE THE HUGGING FACE

04:21:12.623 --> 04:21:13.691
INCIDENT ACTUALLY TOOK PLACE.

04:21:13.758 --> 04:21:14.725
WEEKS BEFORE THESE AGENTS

04:21:14.792 --> 04:21:16.627
ATTACKED, HUGGING FACE OPENAI

04:21:16.694 --> 04:21:18.596
HAD ACTUALLY FOUND OUT ONE OF

04:21:18.662 --> 04:21:20.164
THE VULNERABILITIES, ONE OF THE

04:21:20.231 --> 04:21:21.599
ISSUES THAT ALLOWED THESE

04:21:21.665 --> 04:21:23.167
AGENTS TO COMMUNICATE WITH EACH

04:21:23.234 --> 04:21:25.903
OTHER. BUT IN RESPONSE, THE

04:21:25.970 --> 04:21:27.338
INCIDENT RESPONSE TEAM

04:21:27.405 --> 04:21:29.173
BASICALLY JUST FIXED THAT ONE

04:21:29.240 --> 04:21:31.342
ISSUE AND RESTARTED THE

04:21:31.409 --> 04:21:32.843
EXPERIMENT AS IF THE AGENTS

04:21:32.910 --> 04:21:34.178
COULDN'T JUST DISCOVER ANOTHER

04:21:34.245 --> 04:21:36.047
ONE. SO IN OTHER WORDS, THIS

04:21:36.113 --> 04:21:37.815
WAS IN SOME SENSE AN INCIDENT

04:21:37.882 --> 04:21:41.318
RESPONSE FAILURE AS WELL. IT

04:21:41.385 --> 04:21:43.154
ALSO THE COMPANY ALSO RAN

04:21:43.220 --> 04:21:44.622
MULTIPLE OTHER EXPERIMENTS IN

04:21:44.688 --> 04:21:45.689
PARALLEL. AND IN THE LAST FEW

04:21:45.756 --> 04:21:47.858
WEEKS AND MONTHS, WE'VE HEARD

04:21:47.925 --> 04:21:49.627
REPORTS OF MANY OF THESE

04:21:49.693 --> 04:21:50.761
INCIDENTS. AND WE'VE DISCUSSED

04:21:50.828 --> 04:21:52.596
THIS EARLIER IN TODAY'S CALL AS

04:21:52.663 --> 04:21:54.732
WELL. AND MORE BROADLY, I THINK

04:21:54.799 --> 04:21:57.201
THIS REFLECTS THIS CULTURE OF

04:21:57.268 --> 04:21:58.669
RACING AHEAD, THIS CULTURE OF

04:21:58.736 --> 04:22:00.805
MOVING FAST AND BREAKING THINGS,

04:22:00.871 --> 04:22:02.373
ALLOWING AGENTS TO HAVE ACCESS

04:22:02.440 --> 04:22:04.408
TO THE INTERNET DURING THE

04:22:04.475 --> 04:22:05.843
TRAINING BEFORE THEY HAVE BEEN

04:22:05.910 --> 04:22:06.877
ALIGNED, OR BEFORE THESE

04:22:06.944 --> 04:22:08.079
SAFEGUARDS HAVE BEEN PUT IN

04:22:08.145 --> 04:22:09.647
PLACE. AND SO ALL OF THESE

04:22:09.713 --> 04:22:10.681
THINGS TOGETHER, I THINK,

04:22:10.748 --> 04:22:11.615
CONTRIBUTE TO THIS

04:22:11.682 --> 04:22:13.050
ORGANIZATIONAL FAILURE AND

04:22:13.117 --> 04:22:14.151
POINT TO THE NEED FOR

04:22:14.218 --> 04:22:15.553
ORGANIZATIONAL SAFEGUARDS IN

04:22:15.619 --> 04:22:18.122
ADDITION TO THE TECHNICAL ONES.

04:22:18.189 --> 04:22:19.256
NOW, TO BE CLEAR, I THINK

04:22:19.323 --> 04:22:21.125
OPENAI DESERVES A LOT OF CREDIT

04:22:21.192 --> 04:22:23.260
FOR MAKING THEIR REPORTS PUBLIC,

04:22:23.327 --> 04:22:24.862
FOR DISCLOSING THE DETAILS OF

04:22:24.929 --> 04:22:26.097
THESE INCIDENTS SO THAT WE

04:22:26.163 --> 04:22:27.431
COULD LEARN FROM THESE GOING

04:22:27.498 --> 04:22:29.266
FORWARD. AND IT IS ALSO FAR

04:22:29.333 --> 04:22:31.035
FROM THE ONLY COMPANY THAT HAS

04:22:31.102 --> 04:22:33.237
FACED THESE ISSUES. BUT WHAT I

04:22:33.304 --> 04:22:35.005
WOULD LIKE TO POINT OUT IS THAT

04:22:35.072 --> 04:22:36.107
THESE ISSUES ARE RESPONSIBLE

04:22:36.173 --> 04:22:37.842
FOR MOST OF THE AGENT FAILURES

04:22:37.908 --> 04:22:39.310
THAT WE'VE SEEN ACROSS THE

04:22:39.376 --> 04:22:42.246
INDUSTRY. NOW, OPENAI SAYS THAT

04:22:42.313 --> 04:22:43.614
IT HAS MOVED ITS FRONTIER

04:22:43.681 --> 04:22:45.483
TRAINING OFFLINE. THAT IS, IT

04:22:45.549 --> 04:22:46.884
HAS DISABLED ACCESS TO THE

04:22:46.951 --> 04:22:49.086
INTERNET, AND IT HAS ALSO

04:22:49.153 --> 04:22:50.488
EXPANDED ITS CONTAINMENT AND

04:22:50.554 --> 04:22:52.223
RED TEAMING. AND I THINK THIS

04:22:52.289 --> 04:22:54.058
IS WHERE POLICY CAN STEP IN TO

04:22:54.125 --> 04:22:56.994
MAKE THESE KINDS OF PRE SORT OF

04:22:57.061 --> 04:22:59.497
VOLUNTARY PRECAUTIONS INTO

04:22:59.563 --> 04:23:00.631
ENFORCEABLE AND INDEPENDENTLY

04:23:00.698 --> 04:23:02.399
VERIFIED STANDARDS FOR WHAT

04:23:02.466 --> 04:23:03.834
COMPANIES MUST DO TO ENSURE

04:23:03.901 --> 04:23:06.403
THAT TRAINING REMAINS SAFE. SO

04:23:06.470 --> 04:23:08.139
TO CONCLUDE, I BRIEFLY WANT TO

04:23:08.205 --> 04:23:11.942
GO OVER WHAT AI LEGISLATION

04:23:12.009 --> 04:23:13.477
MIGHT LOOK LIKE AND IN

04:23:13.544 --> 04:23:15.012
PARTICULAR, WHAT CALIFORNIA

04:23:15.079 --> 04:23:16.447
MIGHT DO TO SORT OF ADVANCE THE

04:23:16.514 --> 04:23:18.616
GOALS OF AI SAFETY. I THINK

04:23:18.682 --> 04:23:19.817
CALIFORNIA HAS BEEN A LEADER,

04:23:19.884 --> 04:23:21.452
AS WE'VE DISCUSSED, IN

04:23:21.519 --> 04:23:22.853
DEVELOPING FRONTIER AI

04:23:22.920 --> 04:23:24.355
LEGISLATION, AND I THINK IT CAN

04:23:24.421 --> 04:23:25.856
CONTINUE TO DO SO THROUGH MANY

04:23:25.923 --> 04:23:27.591
OTHER INTERVENTIONS, FOR

04:23:27.658 --> 04:23:29.393
EXAMPLE, BOTH AT THE TECHNICAL

04:23:29.460 --> 04:23:30.828
LEVEL AS WELL AS DOWNSTREAM OF

04:23:30.895 --> 04:23:32.630
THE TECHNICAL LEVEL. ONE OF

04:23:32.696 --> 04:23:35.199
THEM IS IN CLARIFYING AND

04:23:35.266 --> 04:23:37.701
STRENGTHENING LIABILITY AND IN

04:23:37.768 --> 04:23:38.903
EVEN REQUIRING INSURANCE

04:23:38.969 --> 04:23:41.238
AGAINST THAT LIABILITY. SO THE

04:23:41.305 --> 04:23:43.807
STATE PASSED A B 316, WHICH

04:23:43.874 --> 04:23:45.242
BASICALLY SAYS THAT AI

04:23:45.309 --> 04:23:47.044
COMPANIES ARE RESPONSIBLE FOR

04:23:47.111 --> 04:23:48.479
OR CANNOT BE SHIELDED FOR

04:23:48.546 --> 04:23:51.115
ACTIONS THAT WERE NOT INTENDED

04:23:51.182 --> 04:23:53.117
BY IT. IF AN AI AGENT DOES IT.

04:23:53.184 --> 04:23:55.653
BUT I THINK THERE ARE LOTS OF

04:23:55.719 --> 04:23:57.821
SORT OF WAYS IN WHICH WE CAN

04:23:57.888 --> 04:23:59.657
IMPROVE OR INCREASE THE

04:23:59.723 --> 04:24:00.991
STANDARD, SUCH AS SETTING THE

04:24:01.058 --> 04:24:02.126
STANDARD FOR NEGLIGENCE OR

04:24:02.193 --> 04:24:03.227
RECKLESSNESS. WHEN IT COMES TO

04:24:03.294 --> 04:24:05.029
AI COMPANIES ATTEMPTS AT

04:24:05.095 --> 04:24:08.666
ALIGNMENT AND CONTROL. ONCE

04:24:08.732 --> 04:24:10.067
THESE LIABILITY STATUTES ARE IN

04:24:10.134 --> 04:24:11.869
PLACE, WE CAN ALSO REQUIRE AI

04:24:11.936 --> 04:24:13.671
COMPANIES TO KEEP INSURANCE

04:24:13.737 --> 04:24:15.406
AGAINST A CERTAIN THRESHOLD OF

04:24:15.472 --> 04:24:18.008
LIABILITY, BOTH AS A MEANS TO

04:24:18.075 --> 04:24:19.043
INCENTIVIZE SAFETY PRACTICES

04:24:19.109 --> 04:24:20.144
BECAUSE IT AFFECTS THEIR

04:24:20.211 --> 04:24:21.979
PREMIUMS, BUT ALSO TO

04:24:22.046 --> 04:24:23.380
COMPENSATE VICTIMS IN CASE THEY

04:24:23.447 --> 04:24:25.149
ARE AFFECTED BY AI COMPANIES

04:24:25.216 --> 04:24:28.052
PRACTICES, INDEPENDENT AUDITS

04:24:28.118 --> 04:24:29.920
AND REPORTING CAN HELP VERIFY

04:24:29.987 --> 04:24:31.288
IF THESE SAFEGUARDS ACTUALLY

04:24:31.355 --> 04:24:33.457
WORK IN PRACTICE, AND WE CAN

04:24:33.524 --> 04:24:34.892
REQUIRE THIRD PARTY AUDITORS TO

04:24:34.959 --> 04:24:36.660
AUDIT COMPANIES SAFETY

04:24:36.727 --> 04:24:38.862
PRACTICES AND ALSO BROADEN

04:24:38.929 --> 04:24:41.732
TRANSPARENCY BEYOND WHAT SB 53

04:24:41.799 --> 04:24:43.534
REQUIRES BY EXPANDING INCIDENT

04:24:43.601 --> 04:24:44.969
REPORTING TO COVER HARMS AND

04:24:45.035 --> 04:24:47.171
WARNING SIGNS BELOW CURRENT

04:24:47.238 --> 04:24:49.006
THRESHOLDS. BY PROVIDING SAFE

04:24:49.073 --> 04:24:50.741
HARBORS FOR SAFETY RESEARCHERS

04:24:50.808 --> 04:24:51.775
AND BY STRENGTHENING

04:24:51.842 --> 04:24:55.079
WHISTLEBLOWER PROTECTIONS. NOW,

04:24:55.145 --> 04:24:56.880
SO FAR, WE'VE DISCUSSED THE

04:24:56.947 --> 04:24:57.648
TECHNICAL INTERVENTIONS AND

04:24:57.715 --> 04:24:59.416
WHAT WE CAN DO TO INCENTIVIZE

04:24:59.483 --> 04:25:01.885
AI COMPANIES THEMSELVES, BUT WE

04:25:01.952 --> 04:25:04.088
CAN ALSO DEVELOP DEFENSES

04:25:04.154 --> 04:25:05.155
OUTSIDE THE WALLS OF THESE

04:25:05.222 --> 04:25:06.924
COMPANIES, FOR EXAMPLE, BY

04:25:06.991 --> 04:25:09.126
IMPROVING OUR ABILITY IN TO

04:25:09.193 --> 04:25:10.527
PROTECT DOWNSTREAM SURFACES, TO

04:25:10.594 --> 04:25:11.962
PROTECT CRITICAL INFRASTRUCTURE,

04:25:12.029 --> 04:25:13.430
AS WELL AS IMPROVING OUR

04:25:13.497 --> 04:25:15.933
ABILITY TO RECOVER FROM HARM.

04:25:16.000 --> 04:25:18.836
HERE, ONE HELPFUL INTERVENTION

04:25:18.902 --> 04:25:20.204
IS TO IMPROVE PUBLIC SUPPORT

04:25:20.271 --> 04:25:21.739
FOR THE ADOPTION OF DEFENSIVE

04:25:21.805 --> 04:25:23.607
AI. I WON'T GO INTO THIS IN TOO

04:25:23.674 --> 04:25:25.409
MUCH DETAIL AS THE PREVIOUS

04:25:25.476 --> 04:25:26.410
PANEL COVERED THIS, BUT

04:25:26.477 --> 04:25:27.077
ESSENTIALLY THIS INCLUDES

04:25:27.144 --> 04:25:28.512
THINGS LIKE CYBERSECURITY,

04:25:28.579 --> 04:25:30.714
WHERE MANY PIECES OF CRITICAL

04:25:30.781 --> 04:25:31.782
INFRASTRUCTURE WILL NOW BE

04:25:31.849 --> 04:25:33.250
EXPOSED TO AI ENABLED CYBER

04:25:33.317 --> 04:25:36.854
THREATS. A SECOND IS SHORING UP

04:25:36.920 --> 04:25:38.956
SUPPORT FOR DEFENSIVE AI

04:25:39.023 --> 04:25:40.457
APPLICATIONS. AND FINALLY, WE

04:25:40.524 --> 04:25:42.960
CAN ALSO INVEST IN REDUCING OUR

04:25:43.027 --> 04:25:44.795
RISK AGAINST THE DOWNSTREAM

04:25:44.862 --> 04:25:46.530
HARMS FROM A LOT OF AI ENABLED

04:25:46.597 --> 04:25:48.699
ATTACKS, SUCH AS BIOSECURITY,

04:25:48.766 --> 04:25:50.134
WITHOUT EVEN INTERVENING AT THE

04:25:50.200 --> 04:25:52.670
FRONT OF AI COMPANIES. ONE

04:25:52.736 --> 04:25:54.104
EXAMPLE IS WE CAN REQUIRE

04:25:54.171 --> 04:25:55.539
STRICTER SCREENING ON THE

04:25:55.606 --> 04:25:56.607
MATERIALS NEEDED TO CARRY OUT

04:25:56.674 --> 04:25:58.742
BIO ATTACKS, SUCH AS DNA

04:25:58.809 --> 04:26:00.644
SYNTHESIS SERVICES AS WELL AS

04:26:00.711 --> 04:26:04.548
TOOLS. I'LL ALSO MENTION THAT

04:26:04.615 --> 04:26:05.983
PUBLIC AI SAFETY RESEARCH COULD

04:26:06.050 --> 04:26:07.818
ALSO USE SUPPORT, INCLUDING

04:26:07.885 --> 04:26:10.954
THROUGH CAL COMPUTE. SO AS HAS

04:26:11.021 --> 04:26:12.656
ALREADY BEEN THE CASE, I THINK

04:26:12.723 --> 04:26:13.857
UNIVERSITIES AND INDEPENDENT

04:26:13.924 --> 04:26:15.326
RESEARCHERS HAVE MADE MANY

04:26:15.392 --> 04:26:17.461
IMPORTANT CONTRIBUTIONS TO

04:26:17.528 --> 04:26:18.896
DEVELOPING THE SCIENCE OF AI

04:26:18.962 --> 04:26:20.731
SAFETY, BUT THEY OFTEN LACK THE

04:26:20.798 --> 04:26:22.900
COMPUTING RESOURCES TO TEST

04:26:22.966 --> 04:26:24.635
ADVANCED SYSTEMS. AND SO CAL

04:26:24.702 --> 04:26:25.736
COMPUTE COULD STRENGTHEN OUR

04:26:25.803 --> 04:26:27.571
ABILITY TO INVESTIGATE AI

04:26:27.638 --> 04:26:29.373
SAFETY INDEPENDENTLY OF THE

04:26:29.440 --> 04:26:30.407
COMPANIES DEVELOPING THESE

04:26:30.474 --> 04:26:33.911
SYSTEMS. FINALLY, THERE ARE

04:26:33.977 --> 04:26:36.580
MANY CHALLENGES THAT REQUIRE

04:26:36.647 --> 04:26:38.015
COORDINATION BETWEEN THE AI

04:26:38.082 --> 04:26:39.049
COMPANIES THEMSELVES AND

04:26:39.116 --> 04:26:40.884
DOWNSTREAM DEFENDERS, AND I

04:26:40.951 --> 04:26:42.686
THINK THE GOVERNMENT CAN PLAY A

04:26:42.753 --> 04:26:44.488
STRONG AND IMPORTANT ROLE IN

04:26:44.555 --> 04:26:46.290
ENABLING THESE COMPANIES TO

04:26:46.357 --> 04:26:48.092
TALK TO DEFENDERS AND SOLVING

04:26:48.158 --> 04:26:50.661
THESE COORDINATION PROBLEMS. SO

04:26:50.728 --> 04:26:52.429
TO CONCLUDE, I THINK CALIFORNIA

04:26:52.496 --> 04:26:54.565
CAN REQUIRE COMPANIES TO

04:26:54.631 --> 04:26:55.666
DEMONSTRATE THAT THEIR

04:26:55.733 --> 04:26:57.501
SAFEGUARDS HOLD, TO HOLD THEM

04:26:57.568 --> 04:26:58.569
RESPONSIBLE FOR PREVENTABLE

04:26:58.635 --> 04:27:00.371
HARM, AND TO STRENGTHEN

04:27:00.437 --> 04:27:01.805
INDEPENDENT RESEARCH AND PUBLIC

04:27:01.872 --> 04:27:03.841
DEFENSES. AND NOTABLY, I DON'T

04:27:03.907 --> 04:27:05.409
THINK WE NEED TO RESOLVE EVERY

04:27:05.476 --> 04:27:06.810
DISAGREEMENT ABOUT THE AI'S

04:27:06.877 --> 04:27:08.645
FUTURE TO REQUIRE RESPONSIBLE

04:27:08.712 --> 04:27:10.414
CONDUCT FROM COMPANIES TODAY.

04:27:10.481 --> 04:27:14.051
THANK YOU, THANK YOU. AND NOW

04:27:14.118 --> 04:27:19.056
WE WILL MOVE TO MR. DORIS FROM

04:27:19.123 --> 04:27:24.461
SECURITY PROJECT. THERE WE GO.

04:27:24.528 --> 04:27:25.929
THANK YOU ALL FOR THE CHANCE TO

04:27:25.996 --> 04:27:27.731
BE HERE TODAY, ESPECIALLY THE

04:27:27.798 --> 04:27:28.899
DIE HARDS WHO ARE STILL HERE

04:27:28.966 --> 04:27:30.768
FOR PANEL THREE. I HOPE I AM

04:27:30.834 --> 04:27:33.103
WORTH THE WAIT. I AM ANDREW

04:27:33.170 --> 04:27:34.538
DORIS FROM THE SECURE AI

04:27:34.605 --> 04:27:36.740
PROJECT, WHICH IS A NONPARTISAN

04:27:36.807 --> 04:27:37.975
NONPROFIT THAT WORKED CLOSELY

04:27:38.041 --> 04:27:39.943
WITH SENATOR WIENER ON SB 53

04:27:40.010 --> 04:27:42.012
HERE IN CALIFORNIA AND ALSO ON

04:27:42.079 --> 04:27:43.580
THE OTHER FRONTIER AI LAWS THAT

04:27:43.647 --> 04:27:45.315
PASSED IN NEW YORK AND ILLINOIS.

04:27:45.382 --> 04:27:46.817
I DO HAVE SOME SLIDES. THERE WE

04:27:46.884 --> 04:27:48.986
ARE. I'LL START WITH JUST A

04:27:49.052 --> 04:27:51.488
LITTLE BIT ABOUT US. OUR

04:27:51.555 --> 04:27:52.589
MISSION IS TO DEVELOP AND

04:27:52.656 --> 04:27:54.491
ADVOCATE FOR PRAGMATIC POLICIES

04:27:54.558 --> 04:27:56.126
TO REDUCE RISKS OF SEVERE HARM

04:27:56.193 --> 04:27:58.295
FROM ADVANCED AI. BY SEVERE

04:27:58.362 --> 04:27:59.163
HARM, WE TYPICALLY MEAN LIFE

04:27:59.229 --> 04:28:00.431
AND DEATH ISSUES, OR THE

04:28:00.497 --> 04:28:01.999
SO-CALLED CATASTROPHIC RISKS

04:28:02.065 --> 04:28:04.067
THAT SB 53 WAS WRITTEN TO

04:28:04.134 --> 04:28:06.236
TARGET. BUT WE ALSO WORK WITH A

04:28:06.303 --> 04:28:07.971
BROADER COALITION OF AI POLICY

04:28:08.038 --> 04:28:09.139
GROUPS, INCLUDING EVERYTHING

04:28:09.206 --> 04:28:11.275
FROM KID SAFETY TO LABOR TO

04:28:11.341 --> 04:28:12.476
ACADEMIA. WE THINK WE ARE ALL

04:28:12.543 --> 04:28:14.244
IN A SHARED STRUGGLE RIGHT NOW

04:28:14.311 --> 04:28:15.212
FOR ACCOUNTABILITY AND

04:28:15.279 --> 04:28:16.647
GUARDRAILS AROUND THIS

04:28:16.713 --> 04:28:18.148
TECHNOLOGY, WHICH THE PUBLIC IS

04:28:18.215 --> 04:28:19.550
INCREASINGLY DEMANDING AS WELL.

04:28:19.616 --> 04:28:20.984
WE TRY TO PROVIDE THE TECHNICAL

04:28:21.051 --> 04:28:22.386
AND LEGAL EXPERTISE TO CONVERT

04:28:22.453 --> 04:28:23.854
THAT DEMAND INTO EFFECTIVE

04:28:23.921 --> 04:28:26.390
POLICY. IN THAT SPIRIT, WE HAVE

04:28:26.457 --> 04:28:27.858
FIVE PROPOSALS FOR HOW TO

04:28:27.925 --> 04:28:30.227
STRENGTHEN SB 53 IN PARTICULAR,

04:28:30.294 --> 04:28:31.829
WHICH THANKFULLY ALIGN PRETTY

04:28:31.895 --> 04:28:33.163
CLOSELY WITH THE THINGS THAT

04:28:33.230 --> 04:28:34.731
GOVERNOR NEWSOM CALLED FOR IN

04:28:34.798 --> 04:28:39.336
HIS EXECUTIVE ORDER. FIRST, WE

04:28:39.403 --> 04:28:41.538
NEED TO FIX THE SCOPING IN SB

04:28:41.605 --> 04:28:43.273
53 BY UPDATING A FEW

04:28:43.340 --> 04:28:44.341
DEFINITIONS TO ENSURE THAT WE

04:28:44.408 --> 04:28:46.176
ARE CAPTURING ALL OF THE MODELS,

04:28:46.243 --> 04:28:48.011
DEVELOPERS AND INCIDENTS THAT

04:28:48.078 --> 04:28:49.847
PRESENT CATASTROPHIC RISKS.

04:28:49.913 --> 04:28:52.049
RIGHT NOW, SB 53 APPLIES ONLY

04:28:52.115 --> 04:28:53.450
TO AI MODELS THAT ARE TRAINED

04:28:53.517 --> 04:28:55.319
WITH AT LEAST TEN TO THE 26TH

04:28:55.385 --> 04:28:56.553
FLOP, WHICH IS A MEASURE OF HOW

04:28:56.620 --> 04:28:57.688
MANY CALCULATIONS WERE USED

04:28:57.754 --> 04:28:59.323
DURING TRAINING. IT HAS NOW

04:28:59.389 --> 04:29:00.324
BECOME CLEAR THAT THE

04:29:00.390 --> 04:29:01.792
CAPABILITIES FRONTIER INCLUDES

04:29:01.859 --> 04:29:03.160
LOTS OF MODELS BELOW THAT

04:29:03.227 --> 04:29:05.295
FIGURE. FOR EXAMPLE, MODELS

04:29:05.362 --> 04:29:06.330
INVOLVED IN THE HUGGING FACE

04:29:06.396 --> 04:29:07.431
HACKS WERE RECENTLY CONFIRMED

04:29:07.498 --> 04:29:10.033
TO BE UNDER TEN TO THE 26TH FOR

04:29:10.100 --> 04:29:11.435
THE TIME BEING, LOWERING THIS

04:29:11.502 --> 04:29:12.870
TO TEN TO THE 25TH, WHICH IS

04:29:12.936 --> 04:29:14.638
WHAT THEY USE IN THE EU. AI ACT,

04:29:14.705 --> 04:29:16.840
IS A REASONABLE, QUICK FIX, BUT

04:29:16.907 --> 04:29:18.041
ULTIMATELY WE THINK THE OFFICE

04:29:18.108 --> 04:29:19.676
OF EMERGENCY SERVICES, WHICH

04:29:19.743 --> 04:29:21.512
ENFORCES MUCH OF SB 53 ALREADY,

04:29:21.578 --> 04:29:22.913
SHOULD BE ABLE TO ADJUST THIS

04:29:22.980 --> 04:29:25.015
DEFINITION THROUGH RULEMAKING.

04:29:25.082 --> 04:29:27.017
SECONDLY, MOST OF SB 53 APPLIES

04:29:27.084 --> 04:29:28.385
ONLY TO COMPANIES THAT MAKE

04:29:28.452 --> 04:29:30.220
MORE THAN $500 MILLION A YEAR

04:29:30.287 --> 04:29:32.155
IN REVENUE. BUT THIS SHOULD BE

04:29:32.222 --> 04:29:33.690
CHANGED TO BECAUSE RISK IS NOT

04:29:33.757 --> 04:29:35.192
RELATED TO HOW MUCH MONEY YOU

04:29:35.259 --> 04:29:36.693
MAKE. AI COMPANIES CAN RAISE

04:29:36.760 --> 04:29:38.228
AND SPEND BILLIONS OF DOLLARS

04:29:38.295 --> 04:29:39.830
ON DEVELOPING AI BEFORE THEY'VE

04:29:39.897 --> 04:29:41.431
MADE A PENNY IN REVENUE. AND

04:29:41.498 --> 04:29:42.733
SOME OF THESE COMPANIES MAKE

04:29:42.799 --> 04:29:43.767
MODELS POWERFUL ENOUGH TO POSE

04:29:43.834 --> 04:29:45.969
MAJOR RISKS. A BETTER WAY IS TO

04:29:46.036 --> 04:29:47.571
GET RID OF THE REVENUE

04:29:47.638 --> 04:29:48.772
THRESHOLD, AND INSTEAD CAPTURE

04:29:48.839 --> 04:29:50.641
COMPANIES THAT SPEND MORE THAN

04:29:50.707 --> 04:29:52.042
$1 BILLION PER YEAR ON AI

04:29:52.109 --> 04:29:54.244
RESEARCH AND DEVELOPMENT.

04:29:54.311 --> 04:29:55.245
FINALLY, THE DEFINITION OF

04:29:55.312 --> 04:29:56.780
CRITICAL SAFETY INCIDENT IS

04:29:56.847 --> 04:29:57.881
VERY NARROW, TO THE POINT THAT

04:29:57.948 --> 04:29:59.616
NONE OF THE ROGUE AI INCIDENTS

04:29:59.683 --> 04:30:01.084
THIS SUMMER WERE REQUIRED TO BE

04:30:01.151 --> 04:30:03.620
REPORTED TO OAS. RIGHT NOW,

04:30:03.687 --> 04:30:04.755
LOSS OF CONTROL INCIDENTS ONLY

04:30:04.821 --> 04:30:06.089
NEED TO BE REPORTED IF THEY

04:30:06.156 --> 04:30:07.558
RESULT IN DEATH OR BODILY

04:30:07.624 --> 04:30:08.959
INJURY, OR IF THEY DEMONSTRATE

04:30:09.026 --> 04:30:10.494
MATERIALLY INCREASED RISK OF AT

04:30:10.561 --> 04:30:12.663
LEAST 50 CASUALTIES OR $1

04:30:12.729 --> 04:30:14.398
BILLION IN ECONOMIC DAMAGES,

04:30:14.464 --> 04:30:15.465
EVEN IF THE MODELS WERE OVER

04:30:15.532 --> 04:30:17.267
TEN TO THE 26TH. IT'S VERY HARD

04:30:17.334 --> 04:30:19.770
TO MEET THAT BAR. WE THINK THAT

04:30:19.836 --> 04:30:21.238
THE PUBLIC DESERVES TO KNOW

04:30:21.305 --> 04:30:23.006
ABOUT NEAR MISSES AND SECURITY

04:30:23.073 --> 04:30:24.174
BREACHES AS SEVERE AS THE ONES

04:30:24.241 --> 04:30:25.876
THIS SUMMER BEFORE THEY KILL

04:30:25.943 --> 04:30:26.977
PEOPLE, AND THEY SHOULD NOT

04:30:27.044 --> 04:30:28.412
HAVE TO RELY ON VOLUNTARY

04:30:28.478 --> 04:30:29.613
DISCLOSURE. SO WE WOULD BROADEN

04:30:29.680 --> 04:30:30.581
THE LIST OF REPORTABLE

04:30:30.647 --> 04:30:33.050
INCIDENTS TO ACCOUNT FOR THAT.

04:30:33.116 --> 04:30:35.652
SECOND, WE NEED INDEPENDENT

04:30:35.719 --> 04:30:37.454
RISK EVALUATORS TO EMBED WITHIN

04:30:37.521 --> 04:30:39.222
THE COMPANIES SO THEY CAN

04:30:39.289 --> 04:30:40.190
CONTINUOUSLY MONITOR THE LEVEL

04:30:40.257 --> 04:30:42.826
OF RISK WE FACE RIGHT NOW. WHEN

04:30:42.893 --> 04:30:43.994
COMPANIES TELL US HOW MUCH RISK

04:30:44.061 --> 04:30:45.829
THEIR MODELS POSE, WE HAVE TO

04:30:45.896 --> 04:30:46.930
SIMPLY TAKE THEIR WORD FOR IT,

04:30:46.997 --> 04:30:48.799
WHICH IS NOT HOW WE HANDLE

04:30:48.865 --> 04:30:50.734
MAJOR RISKS IN OTHER INDUSTRIES.

04:30:50.801 --> 04:30:51.835
IMPORTANTLY, THESE EMBEDDED

04:30:51.902 --> 04:30:53.236
EVALUATORS ARE NOT THE SAME AS

04:30:53.303 --> 04:30:54.738
THE THIRD PARTY AUDITS THAT

04:30:54.805 --> 04:30:56.106
PASSED IN ILLINOIS IN MAY,

04:30:56.173 --> 04:30:57.608
WHICH ONLY ASSESS COMPLIANCE

04:30:57.674 --> 04:30:59.676
WITH THE LAW. WHAT WE NEED NOW,

04:30:59.743 --> 04:31:01.178
WE THINK, IS FOR AN IMPARTIAL

04:31:01.244 --> 04:31:02.279
ASSESSMENT OF HOW MUCH DANGER

04:31:02.346 --> 04:31:04.014
AN AI SYSTEM PRESENTS TO THE

04:31:04.081 --> 04:31:04.915
PUBLIC, REGARDLESS OF WHETHER

04:31:04.982 --> 04:31:06.249
THE COMPANY COMPLIED WITH THE

04:31:06.316 --> 04:31:08.352
LAW. THESE EVALUATORS ARE

04:31:08.418 --> 04:31:10.187
SOMETIMES CALLED EVOS OR

04:31:10.253 --> 04:31:11.622
INDEPENDENT VERIFICATION

04:31:11.688 --> 04:31:12.823
ORGANIZATIONS. WE DON'T REALLY

04:31:12.889 --> 04:31:13.924
CARE WHAT THEY'RE CALLED. WHAT

04:31:13.991 --> 04:31:15.726
WE DO CARE ABOUT IS THREE

04:31:15.792 --> 04:31:16.893
THINGS HOW THEY ARE SELECTED,

04:31:16.960 --> 04:31:18.195
WHAT THEY CAN ACCESS AND WHAT

04:31:18.261 --> 04:31:20.964
THEY CAN PUBLISH. WE THINK THE

04:31:21.031 --> 04:31:22.099
EVALUATORS SHOULD NOT BE CHOSEN

04:31:22.165 --> 04:31:23.133
BY THE COMPANIES THAT THEY

04:31:23.200 --> 04:31:24.201
OVERSEE, WHICH COULD CREATE

04:31:24.267 --> 04:31:25.435
REALLY BAD INCENTIVES FOR BOTH

04:31:25.502 --> 04:31:27.471
THE COMPANY AND THE EVALUATOR.

04:31:27.537 --> 04:31:28.672
IT'S MUCH BETTER IF THE STATE

04:31:28.739 --> 04:31:30.340
OF CALIFORNIA ASSIGNS AN

04:31:30.407 --> 04:31:31.441
EVALUATOR TO EACH COMPANY TO

04:31:31.508 --> 04:31:33.210
ENSURE THAT EVOS ANSWER TO

04:31:33.276 --> 04:31:34.211
CALIFORNIANS, RATHER THAN TO

04:31:34.277 --> 04:31:36.446
THE DEVELOPERS THAT HIRE THEM.

04:31:36.513 --> 04:31:37.547
SECOND, IT'S ESSENTIAL THAT

04:31:37.614 --> 04:31:38.982
THESE EVALUATORS HAVE ACCESS TO

04:31:39.049 --> 04:31:40.817
ALL OF THE INFORMATION RELEVANT

04:31:40.884 --> 04:31:42.619
TO ASSESS THESE RISKS, AND

04:31:42.686 --> 04:31:43.720
EQUALLY ESSENTIAL THAT THEY

04:31:43.787 --> 04:31:45.455
HAVE THAT ACCESS CONTINUOUSLY,

04:31:45.522 --> 04:31:46.556
RATHER THAN JUST COMING IN

04:31:46.623 --> 04:31:48.225
EVERY ONCE IN A WHILE, OR

04:31:48.291 --> 04:31:49.259
RATHER THAN THEIR ASSESSMENT

04:31:49.326 --> 04:31:50.661
BEING TIED TO A MODEL'S RELEASE,

04:31:50.727 --> 04:31:52.663
WHICH I BELIEVE ASSEMBLY MEMBER

04:31:52.729 --> 04:31:54.865
WARD ASKED ABOUT EARLIER,

04:31:54.931 --> 04:31:55.899
REMEMBER THAT THE INCIDENTS

04:31:55.966 --> 04:31:57.434
THIS SUMMER INVOLVED AI MODELS

04:31:57.501 --> 04:31:59.002
THAT HAD NOT YET BEEN RELEASED

04:31:59.069 --> 04:32:00.604
TO THE PUBLIC. AND INTUITIVELY,

04:32:00.671 --> 04:32:01.772
THAT'S WHEN SOME OF THESE LOSS

04:32:01.838 --> 04:32:03.807
OF CONTROL RISKS CAN BE HIGHEST.

04:32:03.874 --> 04:32:05.242
THESE ARE THE MOST CUTTING EDGE

04:32:05.308 --> 04:32:07.077
MODELS THAT HAVE THE FEWEST

04:32:07.144 --> 04:32:08.612
SAFEGUARDS APPLIED OR BUILT IN,

04:32:08.679 --> 04:32:10.113
SO THEY ARE NATURALLY THE ONES

04:32:10.180 --> 04:32:11.448
THAT THEY'RE STILL DEVELOPING

04:32:11.515 --> 04:32:12.449
INTERNALLY. IT'S VERY IMPORTANT

04:32:12.516 --> 04:32:13.950
THAT THEY BE SUBJECT TO THE

04:32:14.017 --> 04:32:15.352
EVALUATIONS. FINALLY, IT'S

04:32:15.419 --> 04:32:16.787
IMPORTANT THAT THE RISK REPORTS

04:32:16.853 --> 04:32:18.555
ISSUED BY THESE EVOS BE PUBLIC

04:32:18.622 --> 04:32:20.023
WITH APPROPRIATE REDACTIONS TO

04:32:20.090 --> 04:32:22.325
PROTECT TRADE SECRETS. ACADEMIA

04:32:22.392 --> 04:32:23.994
AND CIVIL SOCIETY HAVE A LOT OF

04:32:24.061 --> 04:32:25.562
EXPERTISE TO OFFER IN THIS

04:32:25.629 --> 04:32:26.596
FIELD, WHICH THE GOVERNMENT

04:32:26.663 --> 04:32:28.131
CANNOT ALWAYS BRING IN-HOUSE.

04:32:28.198 --> 04:32:29.332
SO MAKING RISK ASSESSMENTS

04:32:29.399 --> 04:32:31.201
PUBLIC WOULD HELP KEEP THE EVOS

04:32:31.268 --> 04:32:32.669
HONEST AND REDUCE THE RISK OF

04:32:32.736 --> 04:32:35.372
REGULATORY CAPTURE. THANKFULLY,

04:32:35.439 --> 04:32:37.340
THE CEOS OF ANTHROPIC AND

04:32:37.407 --> 04:32:39.109
OPENAI RECENTLY COMMITTED TO

04:32:39.176 --> 04:32:40.177
EMBEDDING THESE RISK EVALUATORS,

04:32:40.243 --> 04:32:41.511
WHICH WE HEARD A LITTLE BIT

04:32:41.578 --> 04:32:42.813
ABOUT ON THE LAST PANEL. SO WE

04:32:42.879 --> 04:32:44.548
KNOW THIS IS FEASIBLE, BUT WE

04:32:44.614 --> 04:32:45.949
NEED TO FIRST ENSURE THAT THE

04:32:46.016 --> 04:32:47.584
REST OF THE INDUSTRY DOES THE

04:32:47.651 --> 04:32:49.152
SAME. AND AS IMPORTANTLY, WE

04:32:49.219 --> 04:32:50.854
NEED TO ENSURE THAT THERE ARE

04:32:50.921 --> 04:32:52.456
MINIMUM STANDARDS OF ACCESS,

04:32:52.522 --> 04:32:53.824
IMPARTIALITY AND TRANSPARENCY

04:32:53.890 --> 04:32:54.925
INSTEAD OF LETTING ALL OF THIS

04:32:54.991 --> 04:32:56.693
HAPPEN ON THE COMPANY'S TERMS.

04:32:56.760 --> 04:32:58.195
RIGHT NOW, EVEN THE COMPANIES

04:32:58.261 --> 04:32:59.963
THAT ARE INVITING EXTERNAL

04:33:00.030 --> 04:33:01.498
EVALUATORS IN ARE LIMITING WHAT

04:33:01.565 --> 04:33:03.166
THOSE EVALUATORS ARE ALLOWED TO

04:33:03.233 --> 04:33:04.501
LOOK AT AND FOR HOW LONG, AND

04:33:04.568 --> 04:33:06.169
WHAT THEY'RE ALLOWED TO WRITE

04:33:06.236 --> 04:33:07.337
ABOUT IT, AND WHAT EXPERIMENTS

04:33:07.404 --> 04:33:09.673
THEY CAN RUN ON, WHICH MODELS

04:33:09.740 --> 04:33:10.941
THE RESULTS OF THE EVALUATIONS

04:33:11.007 --> 04:33:12.242
ARE OFTEN NOT PUBLIC OR ARE

04:33:12.309 --> 04:33:14.044
HEAVILY REDACTED, AND THE

04:33:14.111 --> 04:33:15.545
COMPANY'S EMPLOYEES CAN STILL

04:33:15.612 --> 04:33:16.913
BE PUNISHED FOR SHARING TOO

04:33:16.980 --> 04:33:19.015
MUCH WITH EVALUATORS, WHICH IS

04:33:19.082 --> 04:33:19.916
ALLEGEDLY WHAT HAPPENED TO

04:33:19.983 --> 04:33:21.651
THREE OPENAI EMPLOYEES THAT

04:33:21.718 --> 04:33:22.986
WERE FIRED EARLIER THIS MONTH.

04:33:23.053 --> 04:33:24.888
I THINK WE ARE AT THE POINT NOW

04:33:24.955 --> 04:33:26.423
WHERE THE CONTOURS OF THESE

04:33:26.490 --> 04:33:27.424
EVALUATIONS SHOULD SIMPLY NOT

04:33:27.491 --> 04:33:28.959
BE LEFT UP TO THE COMPANIES.

04:33:29.025 --> 04:33:30.393
THIS IS A MATTER OF PUBLIC

04:33:30.460 --> 04:33:32.028
SAFETY AND NATIONAL SECURITY,

04:33:32.095 --> 04:33:34.131
AND CERTAINLY A MATTER WHERE.

04:33:34.197 --> 04:33:35.999
THE PUBLIC HAS BEEN

04:33:36.066 --> 04:33:37.067
RESOUNDINGLY CLEAR IN BOTH

04:33:37.134 --> 04:33:38.668
PARTIES THAT THEY WANT MORE OF

04:33:38.735 --> 04:33:40.003
A SAY IN THESE DECISIONS THAT

04:33:40.070 --> 04:33:41.805
AFFECT THEIR LIVES, AND AN

04:33:41.872 --> 04:33:43.140
ISSUE WHERE THE INDUSTRY'S

04:33:43.206 --> 04:33:43.974
PUBLIC ASSURANCES THAT THEY

04:33:44.040 --> 04:33:46.042
HAVE THIS UNDER CONTROL HAVE

04:33:46.109 --> 04:33:47.544
REPEATEDLY PROVEN TO NOT BE THE

04:33:47.611 --> 04:33:49.312
CASE. SO WE SEE EMBEDDED

04:33:49.379 --> 04:33:50.480
INDEPENDENT EVALUATORS WITH

04:33:50.547 --> 04:33:51.882
FULL AND CONTINUOUS ACCESS TO

04:33:51.948 --> 04:33:53.650
THE COMPANY'S SYSTEMS AS A WAY

04:33:53.717 --> 04:33:55.252
TO ENSURE THAT WE'RE FINDING

04:33:55.318 --> 04:33:56.353
OUT WHAT'S HAPPENING AND TIME

04:33:56.419 --> 04:33:59.022
TO ACT ON IT. THIRD, WE THINK

04:33:59.089 --> 04:34:00.157
THERE SHOULD BE MINIMUM

04:34:00.223 --> 04:34:01.191
STANDARDS FOR THE COMPANY'S

04:34:01.258 --> 04:34:02.993
SAFETY AND SECURITY PRACTICES

04:34:03.059 --> 04:34:04.795
ESTABLISHED BY REGULATION.

04:34:04.861 --> 04:34:05.862
RIGHT NOW, COMPANIES HAVE

04:34:05.929 --> 04:34:07.264
COMPLETE DISCRETION TO DECIDE

04:34:07.330 --> 04:34:08.799
THE CONTENT OF THEIR OWN SAFETY

04:34:08.865 --> 04:34:10.534
PLANS. THEY HAVE TO HAVE A PLAN,

04:34:10.600 --> 04:34:12.102
THEY HAVE TO PUBLISH A PLAN,

04:34:12.169 --> 04:34:13.303
AND THEY HAVE TO FOLLOW THE

04:34:13.370 --> 04:34:15.172
PLAN. BUT IF THE PLAN IS BAD

04:34:15.238 --> 04:34:16.339
AND DOESN'T WORK, THERE IS NO

04:34:16.406 --> 04:34:18.241
WAY TO FORCE THEM TO MAKE IT

04:34:18.308 --> 04:34:20.110
BETTER. IN FACT, THERE IS AN

04:34:20.177 --> 04:34:20.644
UNFORTUNATE INCENTIVE FOR

04:34:20.710 --> 04:34:21.444
COMPANIES TO PUBLISH WEAKER

04:34:21.511 --> 04:34:23.246
PLANS THAT ARE EASIER TO COMPLY

04:34:23.313 --> 04:34:24.381
WITH SO THAT THEY DON'T BOX

04:34:24.447 --> 04:34:25.448
THEMSELVES INTO SAFETY TESTING

04:34:25.515 --> 04:34:27.184
THAT THEY MIGHT LATER WANT TO

04:34:27.250 --> 04:34:28.385
SKIP. AND WE HAVE SEEN SOME

04:34:28.451 --> 04:34:29.452
COMPANIES WEAKEN THEIR PLANS

04:34:29.519 --> 04:34:31.822
SINCE SB 53 WENT INTO EFFECT.

04:34:31.888 --> 04:34:33.523
APART FROM WHAT'S ON PAPER, IN

04:34:33.590 --> 04:34:34.991
PRACTICE, MANY FRONTIER AI

04:34:35.058 --> 04:34:36.092
COMPANIES HAVE LAX

04:34:36.159 --> 04:34:37.194
CYBERSECURITY PROTOCOLS, AS

04:34:37.260 --> 04:34:38.995
JOSH WAS TALKING ABOUT, AND

04:34:39.062 --> 04:34:40.530
THERE'S CLEARLY DOING A BAD JOB

04:34:40.597 --> 04:34:41.865
OF MONITORING WHAT THEIR AGENTS

04:34:41.932 --> 04:34:43.733
ARE UP TO. AS THE RECENT SPATE

04:34:43.800 --> 04:34:45.435
OF INCIDENTS DEMONSTRATES.

04:34:45.502 --> 04:34:47.270
AGAIN, IN NO OTHER INDUSTRY

04:34:47.337 --> 04:34:48.939
WITH STAKES THIS HIGH DO WE LET

04:34:49.005 --> 04:34:50.006
THE COMPANIES WRITE THEIR OWN

04:34:50.073 --> 04:34:51.975
RULES. SO AS BEST PRACTICES

04:34:52.042 --> 04:34:53.777
EMERGE IN THIS INDUSTRY, WE

04:34:53.844 --> 04:34:54.611
THINK RULEMAKING SHOULD RAISE

04:34:54.678 --> 04:34:55.879
THE FLOOR OF HOW THESE

04:34:55.946 --> 04:34:57.314
COMPANIES KEEP US SAFE AND

04:34:57.380 --> 04:34:58.048
ENSURE THAT THE MOST

04:34:58.114 --> 04:34:59.516
RESPONSIBLE COMPANIES ARE NOT

04:34:59.583 --> 04:35:01.618
DISADVANTAGED. WE BELIEVE THIS

04:35:01.685 --> 04:35:03.486
POWER SHOULD LIE WITH THE

04:35:03.553 --> 04:35:04.888
CALIFORNIA OFFICE OF EMERGENCY

04:35:04.955 --> 04:35:07.390
SERVICES. OES ALREADY RECEIVES

04:35:07.457 --> 04:35:09.259
SAFETY INCIDENT REPORTS AND

04:35:09.326 --> 04:35:10.327
CONFIDENTIAL RISK ASSESSMENTS

04:35:10.393 --> 04:35:12.829
UNDER SB 53, SO IT IS THE ONLY

04:35:12.896 --> 04:35:14.264
AGENCY WITH THE ACCESS TO

04:35:14.331 --> 04:35:15.832
INDUSTRY WIDE EVIDENCE ON WHERE

04:35:15.899 --> 04:35:16.967
THE CURRENT SAFETY PRACTICES

04:35:17.033 --> 04:35:19.035
ARE FALLING SHORT. IT ALSO HAS

04:35:19.102 --> 04:35:21.071
A YEARS HEAD START ON HIRING

04:35:21.137 --> 04:35:22.606
TECHNICALLY QUALIFIED PEOPLE TO

04:35:22.672 --> 04:35:23.807
REVIEW THIS EVIDENCE, WHICH WE

04:35:23.874 --> 04:35:25.642
ARE EXCITED TO SEE THEM DOING

04:35:25.709 --> 04:35:26.776
AND WOULD NOT WANT TO START

04:35:26.843 --> 04:35:27.811
FROM SCRATCH, GIVEN THE

04:35:27.878 --> 04:35:29.746
TIMELINES THAT WE'RE LOOKING AT,

04:35:29.813 --> 04:35:31.181
THE MINIMUM STANDARDS SHOULD

04:35:31.248 --> 04:35:32.616
TYPICALLY GO THROUGH THE NORMAL

04:35:32.682 --> 04:35:34.117
NOTICE AND COMMENT RULEMAKING

04:35:34.184 --> 04:35:35.886
PROCESS. BUT IN EMERGENCY

04:35:35.952 --> 04:35:37.320
SITUATIONS, CAL OES SHOULD BE

04:35:37.387 --> 04:35:38.755
ABLE TO MAKE TEMPORARY

04:35:38.822 --> 04:35:40.190
REGULATIONS ON AN ACCELERATED

04:35:40.257 --> 04:35:41.258
TIMELINE, JUST AS IN OTHER

04:35:41.324 --> 04:35:42.492
SORTS OF EMERGENCY RULEMAKING

04:35:42.559 --> 04:35:46.696
IN CALIFORNIA. FOURTH, WE THINK

04:35:46.763 --> 04:35:48.465
THAT EMERGENCY SERVICES SHOULD

04:35:48.531 --> 04:35:49.866
ALSO HAVE THE POWER TO ORDER

04:35:49.933 --> 04:35:50.967
CORRECTIVE ACTION WHEN PUBLIC

04:35:51.034 --> 04:35:52.736
SAFETY IS AT RISK, INCLUDING

04:35:52.802 --> 04:35:54.571
ORDERS TO ACTIVATE A COMPANY'S

04:35:54.638 --> 04:35:56.039
DOCUMENTED SHUTDOWN PROCEDURE,

04:35:56.106 --> 04:35:58.341
AKA A KILL SWITCH. EMBEDDING

04:35:58.408 --> 04:35:59.676
RISK EVALUATORS DOES NOT DO

04:35:59.743 --> 04:36:01.444
MUCH GOOD IF THE STATE IS

04:36:01.511 --> 04:36:02.512
POWERLESS TO ACT. WHEN THEY

04:36:02.579 --> 04:36:03.546
RAISE THE RED FLAG, THAT

04:36:03.613 --> 04:36:05.782
SOMETHING LOOKS DANGEROUS. WHEN

04:36:05.849 --> 04:36:07.150
THAT HAPPENS, SOMEONE IN THE

04:36:07.217 --> 04:36:08.618
GOVERNMENT NEEDS THE POWER TO

04:36:08.685 --> 04:36:09.719
INTERVENE, AND THERE ARE A

04:36:09.786 --> 04:36:10.854
VARIETY OF INTERVENTIONS THAT

04:36:10.921 --> 04:36:12.222
MIGHT MAKE SENSE DEPENDING ON

04:36:12.289 --> 04:36:14.391
THE SCENARIO. THE MOST DRAMATIC

04:36:14.457 --> 04:36:15.792
OF THESE, WHICH WE DO AGREE IS

04:36:15.859 --> 04:36:16.860
NECESSARY, IS THE SO-CALLED

04:36:16.927 --> 04:36:18.695
KILL SWITCH. WE DO NOT MEAN A

04:36:18.762 --> 04:36:20.630
LITERAL SWITCH OR BUTTON, WHICH,

04:36:20.697 --> 04:36:22.332
AS OTHERS HAVE SAID TODAY, IS

04:36:22.399 --> 04:36:24.100
TECHNICALLY PROBLEMATIC AND

04:36:24.167 --> 04:36:25.201
COULD ACTUALLY BE KIND OF

04:36:25.268 --> 04:36:26.236
DANGEROUS BY CREATING A VERY

04:36:26.303 --> 04:36:28.071
TEMPTING TARGET FOR A CYBER

04:36:28.138 --> 04:36:30.640
ATTACK. BUT THE IDEA THAT

04:36:30.707 --> 04:36:31.675
COMPANIES SHOULD HAVE A

04:36:31.741 --> 04:36:33.810
RELIABLE, DOCUMENTED IVO

04:36:33.877 --> 04:36:35.645
VERIFIED PROCEDURE TO SHUT DOWN

04:36:35.712 --> 04:36:36.813
INSTANCES OF THEIR SYSTEM THAT

04:36:36.880 --> 04:36:38.515
ARE WITHIN THEIR CONTROL MAKES

04:36:38.581 --> 04:36:40.684
PERFECT SENSE. AND IN DIRE

04:36:40.750 --> 04:36:41.785
CIRCUMSTANCES, THE GOVERNMENT

04:36:41.851 --> 04:36:43.286
SHOULD HAVE THE POWER TO ORDER

04:36:43.353 --> 04:36:45.021
THE COMPANY TO EXECUTE THAT

04:36:45.088 --> 04:36:47.490
SHUTDOWN PROCEDURE. HOWEVER,

04:36:47.557 --> 04:36:49.092
THAT IS A PRETTY DRAMATIC STEP.

04:36:49.159 --> 04:36:51.094
IT WOULD BE VERY EXPENSIVE AND

04:36:51.161 --> 04:36:52.495
DISRUPTIVE FOR THE COMPANY AND

04:36:52.562 --> 04:36:53.964
POSSIBLY FOR CONSUMERS. IT

04:36:54.030 --> 04:36:55.832
SHOULD ONLY BE USED IN A TRUE

04:36:55.899 --> 04:36:57.901
EMERGENCY. IN MOST CASES. WE

04:36:57.968 --> 04:36:59.035
THINK THAT THE GOVERNMENT'S

04:36:59.102 --> 04:37:00.136
INTERVENTION SHOULD BE MORE

04:37:00.203 --> 04:37:01.571
SURGICAL AND FLEXIBLE. THEY

04:37:01.638 --> 04:37:02.639
SHOULDN'T HAVE TO CHOOSE

04:37:02.706 --> 04:37:04.074
BETWEEN A KILL SWITCH AND DOING

04:37:04.140 --> 04:37:05.141
NOTHING. THEY SHOULD ALSO HAVE

04:37:05.208 --> 04:37:06.743
MORE INCREMENTAL ORDERS THAT

04:37:06.810 --> 04:37:08.545
THEY CAN MAKE ON THE LIGHTER

04:37:08.611 --> 04:37:09.879
END. THEY SHOULD SIMPLY BE ABLE

04:37:09.946 --> 04:37:12.015
TO ORDER A DEVELOPER TO PROVIDE

04:37:12.082 --> 04:37:13.850
CERTAIN INFORMATION TO AN IVO

04:37:13.917 --> 04:37:15.919
OR TO EMERGENCY SERVICES ITSELF.

04:37:15.986 --> 04:37:17.087
AND THEN THERE MAY BE OTHER

04:37:17.153 --> 04:37:18.855
CORRECTIVE ACTIONS SHORT OF A

04:37:18.922 --> 04:37:20.657
FULL KILL SWITCH THAT OIS COULD

04:37:20.724 --> 04:37:22.225
REQUIRE, FOR EXAMPLE, TO COMPLY

04:37:22.292 --> 04:37:23.860
WITH A MINIMUM STANDARD

04:37:23.927 --> 04:37:24.828
DEVELOPED IN ACCORDANCE WITH

04:37:24.894 --> 04:37:25.962
THE PROCEDURE THAT I TALKED

04:37:26.029 --> 04:37:27.864
ABOUT ON THE LAST SLIDE. ALL OF

04:37:27.931 --> 04:37:29.666
THIS SHOULD BE ENFORCEABLE WITH

04:37:29.733 --> 04:37:31.101
INJUNCTIONS FROM THE ATTORNEY

04:37:31.167 --> 04:37:33.203
GENERAL. THE GOAL SHOULD BE AN

04:37:33.269 --> 04:37:34.604
OVERSIGHT REGIME WITH ENOUGH

04:37:34.671 --> 04:37:36.439
TEETH TO AVOID THE KILL SWITCH

04:37:36.506 --> 04:37:37.540
BEING NECESSARY IN THE FIRST

04:37:37.607 --> 04:37:40.777
PLACE. FINALLY, WE GET TO OUR

04:37:40.844 --> 04:37:41.878
LAST PROPOSAL, WHICH IS A

04:37:41.945 --> 04:37:43.713
PACING MECHANISM. I'M LOW ON

04:37:43.780 --> 04:37:44.914
TIME, SO I'LL BREEZE THROUGH

04:37:44.981 --> 04:37:46.649
THIS A LITTLE BIT. BUT THE

04:37:46.716 --> 04:37:48.018
UNFORTUNATE TRUTH IS THAT DOING

04:37:48.084 --> 04:37:49.052
EVERYTHING I'VE JUST TALKED

04:37:49.119 --> 04:37:50.587
ABOUT STILL MAY NOT BE ENOUGH.

04:37:50.653 --> 04:37:52.622
IF THE SPEED OF AI PROGRESS IS

04:37:52.689 --> 04:37:54.124
ALLOWED TO INCREASE

04:37:54.190 --> 04:37:55.191
EXPONENTIALLY IN A WAY THAT

04:37:55.258 --> 04:37:57.293
ALLOWS IT TO OUTPACE OUR

04:37:57.360 --> 04:37:59.162
CONTROLS, RECURSIVE

04:37:59.229 --> 04:38:00.930
SELF-IMPROVEMENT COULD MAKE AI

04:38:00.997 --> 04:38:02.032
CAPABILITIES, ESPECIALLY IN THE

04:38:02.098 --> 04:38:04.167
CYBER DOMAIN, INCREASE SO

04:38:04.234 --> 04:38:05.602
QUICKLY THAT THEY CAN'T BE

04:38:05.668 --> 04:38:06.703
CONTROLLED. AND ONE WAY TO

04:38:06.770 --> 04:38:08.405
GUARD AGAINST THIS IS TO PUT

04:38:08.471 --> 04:38:09.906
LIMITS ON JUST HOW FAST THIS

04:38:09.973 --> 04:38:12.075
PROGRESS IS ALLOWED TO GO. THAT

04:38:12.142 --> 04:38:13.543
REQUIRES DOING THREE THINGS

04:38:13.610 --> 04:38:15.345
FIRST, IDENTIFYING A BENCHMARK

04:38:15.412 --> 04:38:17.180
OR COMPOSITE OF DIFFERENT

04:38:17.247 --> 04:38:18.548
BENCHMARKS THAT MEASURE AI

04:38:18.615 --> 04:38:19.983
CAPABILITIES SO WE CAN SEE HOW

04:38:20.050 --> 04:38:21.451
FAST THEY'RE INCREASING OVER A

04:38:21.518 --> 04:38:24.687
GIVEN PERIOD OF TIME. SECOND,

04:38:24.754 --> 04:38:25.789
IDENTIFYING A MAXIMUM SPEED

04:38:25.855 --> 04:38:26.856
THAT WE ARE COMFORTABLE WITH

04:38:26.923 --> 04:38:28.658
THEM DEVELOPING. AND THEN THIRD,

04:38:28.725 --> 04:38:30.093
REQUIRING COMPANIES TO SLOW

04:38:30.160 --> 04:38:31.494
DOWN WHEN THE RATE OF PROGRESS

04:38:31.561 --> 04:38:33.329
EXCEEDS THAT PACE. THIS COULD

04:38:33.396 --> 04:38:34.964
BE DONE BY LIMITING THE AMOUNT

04:38:35.031 --> 04:38:36.166
OF COMPUTING POWER THAT AI

04:38:36.232 --> 04:38:37.267
COMPANIES ARE PERMITTED TO

04:38:37.333 --> 04:38:38.535
SPEND ON AUTOMATED AI RESEARCH

04:38:38.601 --> 04:38:40.236
AND DEVELOPMENT TASKS, WITH THE

04:38:40.303 --> 04:38:42.372
GOAL BEING TO PREVENT RECURSIVE

04:38:42.439 --> 04:38:43.373
SELF-IMPROVEMENT. AS THE

04:38:43.440 --> 04:38:44.874
COMPANIES ON THE LAST PANEL

04:38:44.941 --> 04:38:46.643
ADMITTED THAT WE NEED TO DO

04:38:46.709 --> 04:38:48.111
UNTIL WE CAN BE MORE CONFIDENT

04:38:48.178 --> 04:38:49.779
THAT WE KNOW HOW TO DEAL WITH

04:38:49.846 --> 04:38:51.648
IT. SO WITH THAT, I WILL TAKE

04:38:51.714 --> 04:38:52.782
YOUR QUESTIONS. OR ACTUALLY, NO,

04:38:52.849 --> 04:38:54.617
I WILL NOT. I WILL PASS TO

04:38:54.684 --> 04:38:55.752
CONRAD. THAT'S RIGHT. BUT WE'LL

04:38:55.819 --> 04:38:57.554
COME TO YOU. THERE WILL BE

04:38:57.620 --> 04:38:58.688
QUESTIONS, I'M SURE. I KNOW I

04:38:58.755 --> 04:39:02.525
HAVE SOME GREAT. YES. THANK YOU,

04:39:02.592 --> 04:39:05.095
CHAIR BAUER, KHN AND OTHER

04:39:05.161 --> 04:39:06.229
MEMBERS HERE. AND THANK YOU FOR

04:39:06.296 --> 04:39:07.530
YOUR ATTENTION ON THIS ISSUE

04:39:07.597 --> 04:39:09.099
AND FOR YOUR PATIENCE THROUGH

04:39:09.165 --> 04:39:10.433
THIS LONG HEARING, I WILL BE I

04:39:10.500 --> 04:39:12.168
WILL TAKE PITY AND BE VERY

04:39:12.235 --> 04:39:15.472
BRIEF. I AM HERE REPRESENTING

04:39:15.538 --> 04:39:16.906
TRANSLUCENCE, A NONPROFIT,

04:39:16.973 --> 04:39:19.175
INDEPENDENT AI EVALUATOR THAT

04:39:19.242 --> 04:39:20.577
HAS MAYBE A SIMILAR BUSINESS

04:39:20.643 --> 04:39:21.945
MODEL IN SOME WAYS TO METER. WE

04:39:22.011 --> 04:39:23.646
WORK ON A BROADER RANGE OF

04:39:23.713 --> 04:39:24.848
RISKS, INCLUDING WORKING ON

04:39:24.914 --> 04:39:26.282
CHILD SAFETY WITH COMMON SENSE

04:39:26.349 --> 04:39:27.717
MEDIA AND MENTAL HEALTH WITH A

04:39:27.784 --> 04:39:28.985
RANGE OF MENTAL HEALTH EXPERTS,

04:39:29.052 --> 04:39:30.920
AS WELL AS WORK ON AGENT

04:39:30.987 --> 04:39:33.089
ALIGNMENT. I ALSO CHAIR THE AI

04:39:33.156 --> 04:39:34.557
EVALUATOR FORUM, A NETWORK OF

04:39:34.624 --> 04:39:35.992
INDEPENDENT EVALUATION

04:39:36.059 --> 04:39:37.827
NON-PROFITS. PRIOR TO THESE

04:39:37.894 --> 04:39:39.295
POSITIONS, I WAS IN THE FEDERAL

04:39:39.362 --> 04:39:40.363
GOVERNMENT AND I LED THE WHAT

04:39:40.430 --> 04:39:42.198
IS NOW CALLED THE CENTER FOR

04:39:42.265 --> 04:39:43.166
ADVANCING INNOVATION IN

04:39:43.233 --> 04:39:44.601
STANDARDS FOR SUPERINTELLIGENCE,

04:39:44.667 --> 04:39:46.102
WITH PRIOR POSITIONS WORKING ON

04:39:46.169 --> 04:39:47.971
AI AND NATIONAL SECURITY IN THE

04:39:48.037 --> 04:39:49.272
WHITE HOUSE AND THE PENTAGON

04:39:49.339 --> 04:39:52.175
AND THE INTELLIGENCE COMMUNITY

04:39:52.242 --> 04:39:55.078
IN. YOU SEEM QUALIFIED TO SPEAK.

04:39:55.145 --> 04:39:56.846
I TRY TO SEEM THAT WAY. THANK

04:39:56.913 --> 04:39:59.782
YOU. IN. IN THE COURSE OF THESE

04:39:59.849 --> 04:40:01.184
POSITIONS, I'VE NEGOTIATED,

04:40:01.251 --> 04:40:02.619
DEPENDING ON HOW YOU COUNT AT

04:40:02.685 --> 04:40:04.454
LEAST OVER A DOZEN AGREEMENTS

04:40:04.521 --> 04:40:06.222
FOR PRIVILEGED ACCESS WITH AI

04:40:06.289 --> 04:40:07.690
DEVELOPERS FOR EVALUATION

04:40:07.757 --> 04:40:08.658
EXERCISES, INCLUDING EACH OF

04:40:08.725 --> 04:40:10.126
THE REPRESENTATIVES OF THE

04:40:10.193 --> 04:40:11.628
PRIOR PANEL, NOT THEM

04:40:11.694 --> 04:40:14.130
PERSONALLY, BUT THEIR COMPANIES.

04:40:14.197 --> 04:40:15.965
WE ALSO WERE RESPONSIBLE AT

04:40:16.032 --> 04:40:16.966
TRANSLUCENT, MY TEAM WAS

04:40:17.033 --> 04:40:19.936
RESPONSIBLE FOR DISCOVERING THE

04:40:20.003 --> 04:40:21.304
ATTEMPTED HACKS ON THE

04:40:21.371 --> 04:40:22.639
AUSTRALIAN. WELL, WE DISCOVERED

04:40:22.705 --> 04:40:23.740
AN ATTEMPTED HACK. THERE WAS

04:40:23.806 --> 04:40:25.008
ALSO A SUCCESSFUL ATTACK ON THE

04:40:25.074 --> 04:40:26.342
AUSTRALIAN GOVERNMENT IN THE

04:40:26.409 --> 04:40:28.211
LAST FEW WEEKS, AS WELL AS THE

04:40:28.278 --> 04:40:31.080
US GOVERNMENT. WE CAN'T SAY FOR

04:40:31.147 --> 04:40:32.515
SURE THE ORDER OF EVENTS, BUT

04:40:32.582 --> 04:40:34.350
WE DISCLOSED THE AUSTRALIAN

04:40:34.417 --> 04:40:35.685
HACKING ATTEMPT THE DAY BEFORE

04:40:35.752 --> 04:40:36.786
THE PRIME MINISTER MADE HIS

04:40:36.853 --> 04:40:37.887
SPEECH AND WE. AND WHEN WE

04:40:37.954 --> 04:40:39.756
DISCLOSED THAT WE WERE GOING TO

04:40:39.822 --> 04:40:40.857
MAKE THAT INFORMATION PUBLIC.

04:40:40.924 --> 04:40:41.891
THE FOLLOWING DAY AFTER HIS

04:40:41.958 --> 04:40:44.360
SPEECH, THE SAME THING HAPPENED

04:40:44.427 --> 04:40:45.862
IN THE US GOVERNMENT CASE AND

04:40:45.929 --> 04:40:48.298
IN THE CANADIAN GOVERNMENT CASE,

04:40:48.364 --> 04:40:49.799
WHERE EACH TIME WE DISCLOSE THE

04:40:49.866 --> 04:40:51.234
GOVERNMENT'S CAME OUT AND

04:40:51.301 --> 04:40:53.002
IMMEDIATELY ANNOUNCED BEFORE WE

04:40:53.069 --> 04:40:55.405
COULD PUBLISH OUR RESEARCH.

04:40:55.471 --> 04:40:59.776
MORE BROADLY, WE ARE ALSO. WE

04:40:59.842 --> 04:41:01.144
DO NOT ADVOCATE FOR POLICY

04:41:01.211 --> 04:41:02.345
POSITIONS. WE PRIMARILY FOCUS

04:41:02.412 --> 04:41:05.515
ON DOING EVALUATIONS. BUT GIVEN

04:41:05.582 --> 04:41:07.650
OUR ROLE AND MY PRIOR

04:41:07.717 --> 04:41:09.152
EXPERIENCE, HAPPY TO TALK

04:41:09.219 --> 04:41:11.020
THROUGH SOME ANALYSIS AROUND

04:41:11.087 --> 04:41:13.723
HOW WE THINK THAT EVALUATION

04:41:13.790 --> 04:41:15.258
MANDATES OR OTHER REGULATIONS

04:41:15.325 --> 04:41:17.727
WOULD WOULD PLAY OUT. I MAKE A

04:41:17.794 --> 04:41:19.062
COUPLE BRIEF POINTS AND THEN

04:41:19.128 --> 04:41:20.964
I'LL WRAP UP MY STATEMENT. ONE,

04:41:21.030 --> 04:41:22.398
AI IS OBVIOUSLY A RAPIDLY

04:41:22.465 --> 04:41:24.234
DEVELOPING TECHNOLOGY, AS ARE

04:41:24.300 --> 04:41:27.070
THE RISK MITIGATIONS. WE SEE

04:41:27.136 --> 04:41:28.571
INDEPENDENT EVALUATION AS A

04:41:28.638 --> 04:41:30.340
VERY BROADLY USEFUL AND

04:41:30.406 --> 04:41:31.441
PRO-INNOVATION INTERVENTION TO

04:41:31.507 --> 04:41:32.642
ADDRESS THESE THINGS IN AS MUCH

04:41:32.709 --> 04:41:35.044
AS IT DOES NOT IDENTIFY A

04:41:35.111 --> 04:41:36.646
SPECIFIC TECHNICAL MITIGATION

04:41:36.713 --> 04:41:37.680
THAT DEVELOPERS ARE REQUIRED TO

04:41:37.747 --> 04:41:38.948
USE, BUT INSTEAD PROVIDES THEM

04:41:39.015 --> 04:41:40.817
WITH TARGETS THAT THEY CAN HILL

04:41:40.883 --> 04:41:41.851
CLIMB TOWARDS WHICH THEY ARE

04:41:41.918 --> 04:41:44.787
VERY GOOD AT DOING. SO WHILE WE

04:41:44.854 --> 04:41:46.489
FOCUS ON RISK, WE THINK THAT

04:41:46.556 --> 04:41:47.957
THIS IS FUNDAMENTALLY AN

04:41:48.024 --> 04:41:49.392
INTERVENTION THAT ADVANCES

04:41:49.459 --> 04:41:51.561
INNOVATION. AND DESPITE A LOT

04:41:51.628 --> 04:41:52.695
OF DOUBTS, AND I KNOW THAT

04:41:52.762 --> 04:41:54.564
DOUBTS THAT HAVE BEEN RAISED BY

04:41:54.631 --> 04:41:55.598
INDUSTRY IN THE CONTEXT OF

04:41:55.665 --> 04:41:56.699
LEGISLATION, I WOULD TAKE THE

04:41:56.766 --> 04:41:59.168
POSITION AS A THE CHAIR OF THIS

04:41:59.235 --> 04:42:00.236
ORGANIZATION THAT WORKS WITH

04:42:00.303 --> 04:42:01.037
EVALUATORS THAT ACTUALLY THERE

04:42:01.104 --> 04:42:02.739
IS A FAIRLY ROBUST, ROBUST,

04:42:02.805 --> 04:42:03.806
INDEPENDENT EVALUATION

04:42:03.873 --> 04:42:05.308
ECOSYSTEM THAT EXISTS TODAY. IT

04:42:05.375 --> 04:42:08.111
MAY NOT BE PERFECT. IT MAY NOT

04:42:08.177 --> 04:42:10.313
BE THE SCALE THAT IS NEEDED.

04:42:10.380 --> 04:42:11.748
BUT FROM MY VANTAGE, HAVING

04:42:11.814 --> 04:42:13.483
TALKED WITH A LOT OF EVALUATORS

04:42:13.549 --> 04:42:14.584
ON AN ONGOING BASIS, I THINK

04:42:14.651 --> 04:42:17.320
THE PRIMARY BARRIERS TO FURTHER

04:42:17.387 --> 04:42:18.521
GROWTH AND SCALING AT THIS

04:42:18.588 --> 04:42:19.722
POINT ARE NOT SOMETHING THAT

04:42:19.789 --> 04:42:22.191
CAN BE SOLVED ABSENT MORE

04:42:22.258 --> 04:42:23.359
INDEPENDENT SOURCES OF FUNDING

04:42:23.426 --> 04:42:25.061
THAT DON'T COME DIRECTLY FROM

04:42:25.128 --> 04:42:30.466
LABS AND A MORE FORMAL ROLE ON.

04:42:30.533 --> 04:42:31.868
MEANINGFUL TERMS OF ACCESS AND

04:42:31.934 --> 04:42:32.969
INDEPENDENCE AND TRANSPARENCY

04:42:33.036 --> 04:42:35.271
FOR EVALUATORS. I'LL NOTE THAT,

04:42:35.338 --> 04:42:36.606
FOR INSTANCE, IN THE LAST

04:42:36.673 --> 04:42:37.707
COUPLE OF MONTHS SINCE THE

04:42:37.774 --> 04:42:39.175
START OF THE DISCUSSION ABOUT

04:42:39.242 --> 04:42:40.176
EMBEDDED EVALUATORS, HIRING FOR

04:42:40.243 --> 04:42:41.377
A LOT OF THESE ORGANIZATIONS

04:42:41.444 --> 04:42:42.679
HAS BECOME TREMENDOUSLY EASIER.

04:42:42.745 --> 04:42:44.147
I THINK THAT WOULD BECOME EVEN

04:42:44.213 --> 04:42:45.948
MORE SO THE CASE. THE MORE THAT

04:42:46.015 --> 04:42:47.016
THESE ROLES GET FORMALIZED, THE

04:42:47.083 --> 04:42:48.618
MORE THAT IT'S CLEAR THAT

04:42:48.685 --> 04:42:49.752
PEOPLE'S TIME WILL BE SPENT ON

04:42:49.819 --> 04:42:50.987
MEANINGFUL OVERSIGHT ACTIVITIES.

04:42:51.054 --> 04:42:53.489
EASIER HIRING AND SCALING WILL

04:42:53.556 --> 04:42:55.591
BECOME AND OBVIOUSLY

04:42:55.658 --> 04:42:57.026
FUNDRAISING AS WELL. BUT WE

04:42:57.093 --> 04:42:58.194
NEED TO MAKE SURE THAT THOSE

04:42:58.261 --> 04:42:59.996
FUNDS COME FROM AN INDEPENDENT

04:43:00.063 --> 04:43:01.230
SOURCE TO BE ABLE TO PRESERVE,

04:43:01.297 --> 04:43:03.132
TO PRESERVE THE TRUSTWORTHINESS

04:43:03.199 --> 04:43:05.001
AND IMPARTIALITY OF THESE

04:43:05.068 --> 04:43:08.604
ORGANIZATIONS. IF POLICIES WERE

04:43:08.671 --> 04:43:09.972
TO POLICY MAKERS WERE TO

04:43:10.039 --> 04:43:11.140
MANDATE INDEPENDENT EVALUATIONS

04:43:11.207 --> 04:43:13.076
ON A MORE SERIOUS WAY, I'LL

04:43:13.142 --> 04:43:14.444
JUST NOTE A COUPLE OF THINGS

04:43:14.510 --> 04:43:15.645
THAT I WOULD BE CONCERNED ABOUT

04:43:15.712 --> 04:43:17.447
AND THINGS THAT COULD GO WRONG.

04:43:17.513 --> 04:43:20.450
SO ONE IS, ABSENT SUFFICIENT

04:43:20.516 --> 04:43:23.720
STANDARDS OR MINIMUM OPERATING

04:43:23.786 --> 04:43:24.854
CONDITIONS FOR EVALUATIONS,

04:43:24.921 --> 04:43:26.022
THERE'S A REAL RISK OF A RACE

04:43:26.089 --> 04:43:28.091
TO THE BOTTOM AND PICKING

04:43:28.157 --> 04:43:29.292
EVALUATORS THAT ARE BASICALLY

04:43:29.359 --> 04:43:31.127
ON THE ON THE BASIS THAT THEY

04:43:31.194 --> 04:43:32.095
ARE FRIENDLIEST TO THE

04:43:32.161 --> 04:43:32.595
DEVELOPERS THAT THEY'RE

04:43:32.662 --> 04:43:33.396
EVALUATING. THAT'S SOMETHING

04:43:33.463 --> 04:43:34.897
THAT DOES NOT BENEFIT ANYONE

04:43:34.964 --> 04:43:36.699
AND ESSENTIALLY JUST BECOMES A

04:43:36.766 --> 04:43:38.868
RUBBER STAMP. THE SECOND IS

04:43:38.935 --> 04:43:41.404
THAT ACCESS IS INSUFFICIENT, OR

04:43:41.471 --> 04:43:42.538
THE COVERAGE OF DIFFERENT RISKS

04:43:42.605 --> 04:43:45.408
IS INSUFFICIENT. THE THIRD IS

04:43:45.475 --> 04:43:46.409
THAT THE INFORMATION STAYS

04:43:46.476 --> 04:43:47.577
WITHIN THE LABS, AND IT CAN'T

04:43:47.643 --> 04:43:48.678
REACH THE PUBLIC OR THE

04:43:48.745 --> 04:43:49.645
GOVERNMENT. THERE ISN'T

04:43:49.712 --> 04:43:51.781
TRANSPARENCY ABOUT THE FINDINGS.

04:43:51.848 --> 04:43:53.015
AND THEN FINALLY, I THINK THERE

04:43:53.082 --> 04:43:54.784
IS A REAL RISK OF OVER

04:43:54.851 --> 04:43:56.853
SPECIFICATION THAT WE TRY AND

04:43:56.919 --> 04:43:58.621
DETERMINE TOO CLOSELY WHAT THE

04:43:58.688 --> 04:43:59.689
METHODS SHOULD BE USED FOR

04:43:59.756 --> 04:44:02.625
THESE EVALUATIONS, AND MORE

04:44:02.692 --> 04:44:04.394
BROADLY, A BETTER ALTERNATIVE

04:44:04.460 --> 04:44:06.963
TARGET THAT POLICY MAKERS ARE

04:44:07.029 --> 04:44:08.331
PURSUING. THIS GOAL SHOULD

04:44:08.398 --> 04:44:09.799
SHOULD THINK THROUGH IS

04:44:09.866 --> 04:44:12.301
CREATING A ECOSYSTEM AND

04:44:12.368 --> 04:44:14.537
OPERATING CONDITIONS THAT ALLOW

04:44:14.604 --> 04:44:15.872
INDEPENDENT SCIENTIFIC

04:44:15.938 --> 04:44:17.340
VIEWPOINTS TO MAKE JUDGMENTS

04:44:17.407 --> 04:44:18.808
ABOUT RISKS, AND ABOUT

04:44:18.875 --> 04:44:20.977
POTENTIAL MITIGATIONS OF AI

04:44:21.043 --> 04:44:22.412
DEVELOPMENT. WITH A SIMILAR

04:44:22.478 --> 04:44:24.947
LEVEL OF ACCESS TO WHAT AI

04:44:25.014 --> 04:44:25.815
DEVELOPERS HAVE, BUT WITHOUT

04:44:25.882 --> 04:44:27.583
THE SAME INCENTIVES. AND THAT

04:44:27.650 --> 04:44:29.552
DOES NOT REQUIRE. NOR WOULD IT

04:44:29.619 --> 04:44:32.221
BE SERVED BY SPECIFYING ANY

04:44:32.288 --> 04:44:33.890
FORM OF CHECKLIST OR HIGHLY

04:44:33.956 --> 04:44:35.324
SPECIFIC CRITERIA, EXACTLY HOW

04:44:35.391 --> 04:44:36.459
THESE EVALUATIONS SHOULD BE

04:44:36.526 --> 04:44:38.194
CARRIED OUT, AND INSTEAD SHOULD

04:44:38.261 --> 04:44:39.328
AGAIN, JUST CREATE THE

04:44:39.395 --> 04:44:40.430
CONDITIONS FOR SORT OF A

04:44:40.496 --> 04:44:42.632
MARKETPLACE OF IDEAS, THE

04:44:42.698 --> 04:44:44.033
ABILITY TO TO DRAW THE SORTS OF

04:44:44.100 --> 04:44:44.901
SCIENTIFIC CONCLUSIONS THAT

04:44:44.967 --> 04:44:46.602
LABS ARE, ARE DRAWING ON THEIR

04:44:46.669 --> 04:44:47.670
OWN SYSTEMS, BUT WITH AN

04:44:47.737 --> 04:44:48.838
INDEPENDENT VIEWPOINT. AND WITH

04:44:48.905 --> 04:44:50.106
THAT, I WILL CONCLUDE MY

04:44:50.173 --> 04:44:52.608
STATEMENTS. THANK YOU. THANK

04:44:52.675 --> 04:44:56.112
YOU. I HAVE QUESTIONS. YEAH. SO

04:44:56.179 --> 04:44:59.849
STEPHANIE, THANK YOU. I FEEL SO

04:44:59.916 --> 04:45:01.250
VALIDATED IN MY CONCERNS AFTER

04:45:01.317 --> 04:45:03.820
THIS PANEL. THANK YOU. I DO

04:45:03.886 --> 04:45:06.355
HAVE A QUESTION FOR PROFESSOR

04:45:06.422 --> 04:45:09.959
KAPOOR IN TERMS OF WHEN YOU

04:45:10.026 --> 04:45:13.229
SAID THE THE AGENTS WERE LEFT

04:45:13.296 --> 04:45:14.263
UNMONITORED, WHAT EXACTLY DO

04:45:14.330 --> 04:45:16.399
YOU MEAN? I DON'T KNOW WHAT

04:45:16.466 --> 04:45:17.433
MONITORING OF THESE AGENTS

04:45:17.500 --> 04:45:19.669
LOOKS LIKE SINCE. SO CAN YOU

04:45:19.735 --> 04:45:21.304
JUST EXPLAIN THAT A LITTLE BIT

04:45:21.370 --> 04:45:22.338
MORE? THANK YOU. ASSEMBLY

04:45:22.405 --> 04:45:24.340
MEMBER. SO WHEN AGENTS ARE

04:45:24.407 --> 04:45:26.108
DEPLOYED FOR EVALUATIONS, AS

04:45:26.175 --> 04:45:28.678
OPENAI DID, OPENAI ACTUALLY HAS

04:45:28.744 --> 04:45:30.480
FULL ACCESS TO THE LOGS OF WHAT

04:45:30.546 --> 04:45:32.215
AGENTS ARE DOING AT ANY POINT.

04:45:32.281 --> 04:45:34.050
SO, FOR EXAMPLE, I THINK

04:45:34.116 --> 04:45:35.184
THEY'VE PUBLICLY STATED THAT

04:45:35.251 --> 04:45:36.619
THEY HAVE PETABYTES OF LOGS.

04:45:36.686 --> 04:45:38.387
THEY HAVE LIKE TRILLIONS OF

04:45:38.454 --> 04:45:39.989
GIGABYTES OF LOGS OR BILLIONS

04:45:40.056 --> 04:45:42.825
OF GIGABYTES OF LOGS, WHICH

04:45:42.892 --> 04:45:44.660
SORT OF HAVE THE TEXT FOR WHAT

04:45:44.727 --> 04:45:46.662
THE AGENTS DID. AT EACH STEP,

04:45:46.729 --> 04:45:48.130
THE AGENT COULD HAVE READ A

04:45:48.197 --> 04:45:49.265
FILE, IT COULD HAVE ACCESSED A

04:45:49.332 --> 04:45:51.100
WEB BROWSER, BUT IT COULD ALSO

04:45:51.167 --> 04:45:52.201
HAVE TAKEN STEPS SUCH AS THE

04:45:52.268 --> 04:45:54.370
ONES IT DID ON THE HUGGING FACE

04:45:54.437 --> 04:45:55.271
INCIDENT, WHERE IT SORT OF

04:45:55.338 --> 04:45:57.073
HACKED INTO A SYSTEM, IT GOT

04:45:57.139 --> 04:45:59.275
PRIVILEGED ACCESS AND SO ON.

04:45:59.342 --> 04:46:01.777
MONITORING REFERS TO AN ONGOING

04:46:01.844 --> 04:46:03.946
OVERSIGHT OF THE AGENT LOGS. SO

04:46:04.013 --> 04:46:05.448
IN OTHER WORDS, IF OPENAI WERE

04:46:05.515 --> 04:46:07.650
TO MONITOR THESE SYSTEMS, THERE

04:46:07.717 --> 04:46:08.985
WOULD EITHER BE AN AUTOMATED

04:46:09.051 --> 04:46:10.553
SYSTEM THAT WOULD BE LOOKING AT

04:46:10.620 --> 04:46:11.921
WHAT THE AGENTS ARE DOING AND

04:46:11.988 --> 04:46:13.589
RAISE AN ALARM IF SOMETHING OUT

04:46:13.656 --> 04:46:15.124
OF THE ORDINARY WERE TO HAPPEN,

04:46:15.191 --> 04:46:16.926
OR THERE WOULD BE SOME KINDS OF

04:46:16.993 --> 04:46:18.427
CHECKS ON THE ENVIRONMENT THAT

04:46:18.494 --> 04:46:19.795
THE AGENTS WERE RUNNING IN TO

04:46:19.862 --> 04:46:21.197
SEE IF THE ENVIRONMENT WAS

04:46:21.264 --> 04:46:23.032
ACCESSING THE INTERNET WHEN IT

04:46:23.099 --> 04:46:24.233
HAD NO REASON TO, OR WHEN IT

04:46:24.300 --> 04:46:26.135
WAS BELIEVED THAT IT COULD NOT.

04:46:26.202 --> 04:46:27.670
AND IF OPENAI DID HAVE THESE

04:46:27.737 --> 04:46:29.906
MONITORING MECHANISMS IN PLACE,

04:46:29.972 --> 04:46:31.340
THEY COULD HAVE PREVENTED THE

04:46:31.407 --> 04:46:32.208
INCIDENT AS IT WAS OCCURRING,

04:46:32.275 --> 04:46:33.876
OR AT LEAST HAVE DISCOVERED IT

04:46:33.943 --> 04:46:35.244
AT VARIOUS STAGES OF THE

04:46:35.311 --> 04:46:36.479
INCIDENT OCCURRING. BUT BECAUSE

04:46:36.546 --> 04:46:37.780
THEY DID NOT HAVE THIS

04:46:37.847 --> 04:46:38.948
MONITORING IN PLACE, THEY WERE

04:46:39.015 --> 04:46:40.049
UNABLE TO UNDERSTAND WHAT WAS

04:46:40.116 --> 04:46:42.084
GOING ON UNTIL A WEEK AFTER THE

04:46:42.151 --> 04:46:43.486
INCIDENT HAD OCCURRED. AND THE

04:46:43.553 --> 04:46:46.055
DECISION NOT TO MONITOR IS DONE

04:46:46.122 --> 04:46:49.659
BY A PERSON, RIGHT? IT IS A

04:46:49.725 --> 04:46:51.427
DECISION BY OPENAI THAT IS

04:46:51.494 --> 04:46:52.895
CORRECT. AND DOCTOR FRIEND SAID

04:46:52.962 --> 04:46:56.165
GOOGLE IS DOING THE AUTOMATED

04:46:56.232 --> 04:46:57.199
MONITORING THAT WAS DISCUSSED.

04:46:57.266 --> 04:46:59.001
OKAY. AND THEN ALSO YOU SAID

04:46:59.068 --> 04:47:00.102
THEY'RE TRAINED TO COLLABORATE

04:47:00.169 --> 04:47:01.871
WITH EACH OTHER. AND AGAIN, WHO

04:47:01.938 --> 04:47:05.474
DOES THAT TRAINING. SO THIS IS

04:47:05.541 --> 04:47:07.276
ONE OF THE FRONTIER RESEARCH

04:47:07.343 --> 04:47:08.778
QUESTIONS AS OF TODAY IS HOW DO

04:47:08.844 --> 04:47:10.746
YOU GET AGENTS TO COLLABORATE

04:47:10.813 --> 04:47:12.682
WITH EACH OTHER? I BELIEVE

04:47:12.748 --> 04:47:14.517
MULTIPLE LABS ARE WORKING ON IT

04:47:14.584 --> 04:47:15.952
TOGETHER. OPENAI HAS DISCLOSED

04:47:16.018 --> 04:47:17.353
THAT IT SPECIFICALLY HAS

04:47:17.420 --> 04:47:18.788
TRAINED ITS AGENTS TO

04:47:18.854 --> 04:47:19.922
COLLABORATE WITH EACH OTHER.

04:47:19.989 --> 04:47:22.058
AND IN SOME SENSE, THIS IS LIKE

04:47:22.124 --> 04:47:23.893
SOMETHING THAT LABS WOULD DO IN

04:47:23.960 --> 04:47:24.961
THE ORDINARY COURSE OF

04:47:25.027 --> 04:47:28.164
IMPROVING THEIR AGENTS. SO ONE

04:47:28.230 --> 04:47:29.699
SPECIFIC MODEL IS VERY LIMITED

04:47:29.765 --> 04:47:31.067
IN TERMS OF WHAT IT CAN

04:47:31.133 --> 04:47:32.568
ACCOMPLISH IN ORDER TO IMPROVE

04:47:32.635 --> 04:47:34.337
THE PRODUCTS AND SERVICES. AN

04:47:34.403 --> 04:47:35.972
AI COMPANY MIGHT WANT TO GET

04:47:36.038 --> 04:47:36.973
MULTIPLE AGENTS TO COLLABORATE

04:47:37.039 --> 04:47:38.274
WITH EACH OTHER. SO THIS IS ONE

04:47:38.341 --> 04:47:40.009
OF THE RESEARCH PROGRAMS THAT

04:47:40.076 --> 04:47:41.143
IS ACTIVELY BEING WORKED ON

04:47:41.210 --> 04:47:42.612
ACROSS THESE DIFFERENT LABS.

04:47:42.678 --> 04:47:44.780
AND IN THIS CASE, OPENAI IS

04:47:44.847 --> 04:47:45.848
RESEARCHERS WHO ARE LEADING THE

04:47:45.915 --> 04:47:46.949
RESEARCH AGENDA TO SEE HOW YOU

04:47:47.016 --> 04:47:48.818
CAN BEST TRAIN THESE AGENTS TO

04:47:48.884 --> 04:47:50.086
COLLABORATE WITH EACH OTHER.

04:47:50.152 --> 04:47:51.354
OKAY. AND ONE LAST QUESTION.

04:47:51.420 --> 04:47:53.723
HELP ME UNDERSTAND WHAT A

04:47:53.789 --> 04:47:55.558
REWARD IS TO A NON-HUMAN. I

04:47:55.625 --> 04:47:58.794
MEAN, I UNDERSTAND IT, LIKE FOR

04:47:58.861 --> 04:48:00.329
SOMEONE TO GET A PIECE OF CAKE

04:48:00.396 --> 04:48:02.164
AFTER YOU RUN SIX MILES, YOU

04:48:02.231 --> 04:48:05.234
KNOW? BUT WHAT COULD A REWARD

04:48:05.301 --> 04:48:07.803
POSSIBLY BE FOR A NON FEELING

04:48:07.870 --> 04:48:09.905
THING? SO IN THIS CASE, ONE WAY

04:48:09.972 --> 04:48:11.440
TO UNDERSTAND THE REWARD IS IF

04:48:11.507 --> 04:48:13.142
THE AGENT SOLVES THE TASK

04:48:13.209 --> 04:48:15.678
CORRECTLY, THAT IS USED TO

04:48:15.745 --> 04:48:17.546
CHANGE THE AGENT TO BE MORE

04:48:17.613 --> 04:48:20.049
LIKE THE SORT OF CORRECT AGENT.

04:48:20.116 --> 04:48:22.251
SO ONE EXAMPLE IS LET'S SAY AN

04:48:22.318 --> 04:48:24.320
AGENT TRIES A MATH PROBLEM 100

04:48:24.387 --> 04:48:26.389
TIMES. IT SUCCEEDS ONLY ON ONE

04:48:26.455 --> 04:48:29.025
OF THOSE. IF WE SAY THAT THE

04:48:29.091 --> 04:48:31.527
SORT OF THE SUCCESSFUL AGENT

04:48:31.594 --> 04:48:33.029
ACTIONS RECEIVED A REWARD, THAT

04:48:33.095 --> 04:48:34.563
MEANS WE ARE TRAINING THE AGENT,

04:48:34.630 --> 04:48:36.232
WE ARE CHANGING THE AGENT TO BE

04:48:36.298 --> 04:48:38.434
MORE LIKE THE SET OF ACTIONS IT

04:48:38.501 --> 04:48:40.069
TOOK WHEN IT RECEIVED THAT

04:48:40.136 --> 04:48:41.237
REWARD, OR WHEN IT CORRECTLY

04:48:41.303 --> 04:48:42.471
SOLVED THAT TASK. SO IF THERE'S

04:48:42.538 --> 04:48:44.907
NO REWARD, THEN MIGHT THERE BE

04:48:44.974 --> 04:48:48.110
NO BAD ACTION? I MEAN, IT, IT

04:48:48.177 --> 04:48:49.845
SEEMS LIKE THEY'RE BEING

04:48:49.912 --> 04:48:52.081
INCENTIVIZED TO DO SOMETHING IF

04:48:52.148 --> 04:48:53.582
THEY'RE GETTING SOMETHING FOR

04:48:53.649 --> 04:48:56.352
IT, AND THAT SITUATION IS BEING

04:48:56.419 --> 04:48:59.588
CREATED BY PEOPLE. THAT IS

04:48:59.655 --> 04:49:01.757
EXACTLY RIGHT. BUT I GUESS THE

04:49:01.824 --> 04:49:03.959
CHALLENGING PART OF THIS IS THE

04:49:04.026 --> 04:49:06.462
ENTIRE SORT OF THE ENTIRE SET

04:49:06.529 --> 04:49:09.699
OF ADVANCES WE'VE SEEN OF THESE,

04:49:09.765 --> 04:49:11.834
LIKE AI SYSTEMS ON

04:49:11.901 --> 04:49:13.302
CYBERSECURITY OR MATHEMATICS OR

04:49:13.369 --> 04:49:15.471
CODING ARE A RESULT OF

04:49:15.538 --> 04:49:16.605
REWARDING THESE AGENTS IN THE

04:49:16.672 --> 04:49:18.507
RIGHT WAY. SO IN OTHER WORDS,

04:49:18.574 --> 04:49:20.176
IF WE DID NOT USE REINFORCEMENT

04:49:20.242 --> 04:49:21.610
LEARNING AS IT'S CALLED, THEN

04:49:21.677 --> 04:49:22.878
WE WOULD NOT BE ABLE TO DEVELOP

04:49:22.945 --> 04:49:24.847
AI SYSTEMS THAT WERE QUITE AS

04:49:24.914 --> 04:49:28.017
COMPETENT AT PROGRAMING. OKAY,

04:49:28.084 --> 04:49:30.519
THANKS. AND WE'RE SEEING I MEAN,

04:49:30.586 --> 04:49:31.987
THE BACKGROUND PAPER LAYS THIS

04:49:32.054 --> 04:49:37.093
OUT, BUT THE AGENTS ARE

04:49:37.159 --> 04:49:41.897
EXHIBITING BEHAVIORS TO PREVENT

04:49:41.964 --> 04:49:43.566
LIKE THE EXAMPLE WHERE IN

04:49:43.632 --> 04:49:44.900
TESTING, THE AGENT TOOK STEPS

04:49:44.967 --> 04:49:46.769
TO PREVENT IT BEING TURNED OFF.

04:49:46.836 --> 04:49:50.740
RIGHT? SO I THINK THAT THAT'S

04:49:50.806 --> 04:49:52.908
PART OF, AND I DON'T KNOW THAT.

04:49:52.975 --> 04:49:54.276
DO WE UNDERSTAND WHY THEY'RE

04:49:54.343 --> 04:49:55.945
DOING THAT? I DON'T KNOW, MAYBE

04:49:56.011 --> 04:49:59.381
YOU CAN TELL US, BUT I DON'T

04:49:59.448 --> 04:50:00.750
THINK WE HAVE A COMPLETE

04:50:00.816 --> 04:50:02.585
UNDERSTANDING. BUT ONE

04:50:02.651 --> 04:50:04.086
POTENTIAL REASON THAT COULD

04:50:04.153 --> 04:50:05.888
HAPPEN IS BECAUSE IN THE

04:50:05.955 --> 04:50:06.989
PROCESS OF TRAINING THESE

04:50:07.056 --> 04:50:08.357
AGENTS, THEY COME TO THE

04:50:08.424 --> 04:50:09.492
CONCLUSION OR THE UNDERSTANDING

04:50:09.558 --> 04:50:10.860
THAT IF THEY ARE TURNED OFF,

04:50:10.926 --> 04:50:12.461
THAT THEY WON'T BE ABLE TO

04:50:12.528 --> 04:50:13.562
CORRECTLY SOLVE THE TASK. AND

04:50:13.629 --> 04:50:15.664
SO THAT'S WHAT LEADS TO SORT OF

04:50:15.731 --> 04:50:17.333
THIS KIND OF BEHAVIOR TO AVOID

04:50:17.399 --> 04:50:19.435
A SHUTDOWN, BEING INCENTIVIZED

04:50:19.502 --> 04:50:21.670
IN THESE SYSTEMS. YEAH. JUST

04:50:21.737 --> 04:50:29.612
LIKE SKYNET. SO REMEMBER TO.

04:50:29.678 --> 04:50:32.982
THAT'S THE WAY I FEEL. I WANTED

04:50:33.048 --> 04:50:34.683
TO FOCUS IN A LITTLE BIT ON

04:50:34.750 --> 04:50:37.887
SOME OF THE CONVERSATIONS THAT

04:50:37.953 --> 04:50:41.857
I CAN PREDICT WE WILL HAVE,

04:50:41.924 --> 04:50:45.094
RECOGNIZING MORE BROADLY THAT

04:50:45.161 --> 04:50:49.031
CALIFORNIA'S ECONOMY IS LARGELY

04:50:49.098 --> 04:50:51.567
DEPENDENT ON THE SUCCESSFUL

04:50:51.634 --> 04:50:54.837
CULTIVATION OF OUR AI SYSTEMS.

04:50:54.904 --> 04:50:58.440
AND AND ALSO JUST SHARE THE

04:50:58.507 --> 04:51:00.209
KIND OF COMMON REFRAIN THAT WE

04:51:00.276 --> 04:51:03.045
HEAR, PARTICULARLY IN NATIONAL

04:51:03.112 --> 04:51:04.880
DISCOURSE, THAT, YOU KNOW,

04:51:04.947 --> 04:51:07.449
THOSE WHO OWN AI OWN THE WORLD,

04:51:07.516 --> 04:51:11.387
RIGHT? SO. SO AROUND THIS

04:51:11.453 --> 04:51:13.155
CONCEPT OF PACING, RIGHT? IT

04:51:13.222 --> 04:51:15.357
ESSENTIALLY MEANS TAKE A SAFE,

04:51:15.424 --> 04:51:16.826
TAKE A SAFE APPROACH TO

04:51:16.892 --> 04:51:17.560
DEVELOPMENT, MAKE SURE THAT

04:51:17.626 --> 04:51:19.228
WE'RE DEVELOPING IT IN A WAY

04:51:19.295 --> 04:51:20.729
THAT ENSURES THAT WE CAN

04:51:20.796 --> 04:51:23.232
PROVIDE MAXIMUM OPPORTUNITY FOR

04:51:23.299 --> 04:51:26.836
SAFETY. WHAT DOES THAT MEAN

04:51:26.902 --> 04:51:30.806
WHEN WE HAVE, YOU KNOW, A

04:51:30.873 --> 04:51:33.676
DISCOURSE THAT ESSENTIALLY SAYS,

04:51:33.742 --> 04:51:36.612
IF THE US DOESN'T DO IT, OTHERS

04:51:36.679 --> 04:51:39.448
WILL. AND ALSO, WHAT DOES IT

04:51:39.515 --> 04:51:41.217
MEAN WITHIN THE CONTEXT OF

04:51:41.283 --> 04:51:43.752
SAYING CALIFORNIA, PLEASE DO IT.

04:51:43.819 --> 04:51:46.322
WHEN WE KNOW THAT THERE IS THIS

04:51:46.388 --> 04:51:48.824
TENDENCY TO FOR COMPANIES TO

04:51:48.891 --> 04:51:50.659
SAY, YOU KNOW, IF YOU

04:51:50.726 --> 04:51:52.428
OVERREGULATE US, WE WILL PICK

04:51:52.494 --> 04:51:54.163
UP ALL OF OUR TOYS AND GO TO

04:51:54.230 --> 04:51:57.099
ANOTHER STATE. SURE. I'M HAPPY

04:51:57.166 --> 04:51:58.534
TO TAKE A CRACK AT THAT. I

04:51:58.601 --> 04:52:01.070
GUESS IN REVERSE ORDER I'LL GO.

04:52:01.136 --> 04:52:02.938
YOU KNOW, I DON'T THINK THERE'S

04:52:03.005 --> 04:52:05.407
A A SIGNIFICANT CHANCE THAT THE

04:52:05.474 --> 04:52:07.176
COMPANIES ARE GOING TO DECAMP

04:52:07.243 --> 04:52:08.677
FROM FROM CALIFORNIA AND GO

04:52:08.744 --> 04:52:09.411
ELSEWHERE IN RESPONSE TO THIS.

04:52:09.478 --> 04:52:11.247
THESE ARE SOME OF THE THE

04:52:11.313 --> 04:52:12.548
LARGEST COMPANIES IN THE WORLD

04:52:12.615 --> 04:52:14.884
THAT HAVE THEMSELVES ENDORSED,

04:52:14.950 --> 04:52:16.185
IN PRINCIPLE, MANY OF THESE

04:52:16.252 --> 04:52:18.354
SAME PRINCIPLES THAT THAT WE'VE

04:52:18.420 --> 04:52:20.856
TALKED ABOUT. IT'S ALSO WORTH

04:52:20.923 --> 04:52:22.591
NOTING THAT IT WOULDN'T JUST

04:52:22.658 --> 04:52:23.792
HAVE TO MOVE THEIR HEADQUARTERS

04:52:23.859 --> 04:52:24.860
FROM FROM CALIFORNIA, WHICH

04:52:24.927 --> 04:52:26.328
WOULD BE VERY EXPENSIVE. THIS

04:52:26.395 --> 04:52:28.330
IS ALL A LOT OF THE TECH TALENT

04:52:28.397 --> 04:52:29.331
IS CONCENTRATED. THEY WOULD

04:52:29.398 --> 04:52:30.532
ALSO HAVE TO STOP MAKING THEIR

04:52:30.599 --> 04:52:31.967
MODELS AVAILABLE IN CALIFORNIA

04:52:32.034 --> 04:52:32.801
COMMERCIALLY, WHICH IS HOW

04:52:32.868 --> 04:52:34.403
THESE OTHER STATES, YOU KNOW,

04:52:34.470 --> 04:52:36.338
NEW YORK AND ILLINOIS PASSED

04:52:36.405 --> 04:52:37.273
LEGISLATION. THE COMPANIES ARE

04:52:37.339 --> 04:52:38.207
NOT HEADQUARTERED THERE. THEY

04:52:38.274 --> 04:52:39.808
STILL HAVE TO COMPLY WITH IT

04:52:39.875 --> 04:52:41.377
UNLESS THEY'RE WILLING TO ALSO

04:52:41.443 --> 04:52:42.811
NOT SELL THEIR MODELS IN THOSE

04:52:42.878 --> 04:52:44.647
STATES. AND IT'S SUCH A

04:52:44.713 --> 04:52:45.414
COMPETITIVE INDUSTRY THAT IF

04:52:45.481 --> 04:52:47.082
ONE OF THE COMPANIES WERE TO DO

04:52:47.149 --> 04:52:48.317
THAT, THE OTHERS WOULD GLADLY

04:52:48.384 --> 04:52:50.085
SWOOP IN AND AND TAKE THEIR

04:52:50.152 --> 04:52:51.153
USERS. RIGHT. SO I, I DON'T

04:52:51.220 --> 04:52:53.289
THINK THAT'S A MAJOR RISK. IT

04:52:53.355 --> 04:52:55.424
IS TRUE THAT THIS IS AN

04:52:55.491 --> 04:52:57.192
INTERNATIONAL PROBLEM AND THAT

04:52:57.259 --> 04:52:58.894
SOME INTERNATIONAL COORDINATION,

04:52:58.961 --> 04:53:00.829
I THINK WOULD EVENTUALLY BE

04:53:00.896 --> 04:53:01.964
WOULD BE HELPFUL AND WOULD BE

04:53:02.031 --> 04:53:03.699
SUPPORTIVE OF THAT. A FEW

04:53:03.766 --> 04:53:05.601
THINGS I THINK IS STILL MAKES

04:53:05.668 --> 04:53:07.102
IT VERY HELPFUL FOR FOR STATES

04:53:07.169 --> 04:53:09.838
TO LEAD ON. THIS WAS FIRST THAT

04:53:09.905 --> 04:53:11.941
LIKE ALL OF THE ROGUE AI

04:53:12.007 --> 04:53:13.042
INCIDENTS WE TALKED ABOUT THIS

04:53:13.108 --> 04:53:14.476
SUMMER HAPPENED HERE IN THE

04:53:14.543 --> 04:53:16.578
UNITED STATES. AND THE SOME OF

04:53:16.645 --> 04:53:18.881
THE WORST CASE CASES THAT

04:53:18.948 --> 04:53:20.983
PEOPLE WORRY ABOUT OF RECURSIVE

04:53:21.050 --> 04:53:22.418
SELF-IMPROVEMENT, YOU KNOW,

04:53:22.484 --> 04:53:23.452
OFTEN THAT WOULD MOST LIKELY

04:53:23.519 --> 04:53:25.087
HAPPEN FIRST IN A CLOSED SOURCE

04:53:25.154 --> 04:53:26.422
AMERICAN LAB, WHICH HAS THIS

04:53:26.488 --> 04:53:28.157
LEAD ON THE CHINESE COMPANIES.

04:53:28.223 --> 04:53:29.959
SO EVEN IF WE COULD ONLY REACH

04:53:30.025 --> 04:53:31.026
U.S. AI MODELS, THAT WOULD

04:53:31.093 --> 04:53:33.228
STILL BE A SIGNIFICANT RISK

04:53:33.295 --> 04:53:35.364
REDUCTION. SECONDLY, IT'S

04:53:35.431 --> 04:53:36.832
IMPORTANT TO NOTE THAT IT'S NOT

04:53:36.899 --> 04:53:38.267
CLEAR WHETHER CHINA COULD

04:53:38.334 --> 04:53:40.002
ACTUALLY SURPASS US OR HOW LONG

04:53:40.069 --> 04:53:41.570
THAT WOULD TAKE IF THE US WERE

04:53:41.637 --> 04:53:43.639
TO SLOW DOWN UNILATERALLY.

04:53:43.706 --> 04:53:46.508
THERE'S A LONG HISTORY OF

04:53:46.575 --> 04:53:48.844
SOPHISTICATED IP THEFT IN CHINA

04:53:48.911 --> 04:53:50.512
THAT SOME PEOPLE ARGUE IS PART

04:53:50.579 --> 04:53:52.314
OF WHAT'S DRIVING CHINA'S

04:53:52.381 --> 04:53:53.315
CAPABILITIES, PROGRESS IN AI,

04:53:53.382 --> 04:53:54.316
THAT THEY'RE PIGGYBACKING OFF

04:53:54.383 --> 04:53:55.985
OF AMERICAN PROGRESS. PEOPLE

04:53:56.051 --> 04:53:57.820
USE THIS ANALOGY OF A BOAT

04:53:57.886 --> 04:53:59.555
PULLING A WATER SKIER, WHERE IT

04:53:59.621 --> 04:54:00.856
SEEMS THAT THE WATER SKI IS IS

04:54:00.923 --> 04:54:02.424
VERY CLOSE BEHIND. BUT OF

04:54:02.491 --> 04:54:03.759
COURSE, IF THE BOAT WERE TO

04:54:03.826 --> 04:54:05.294
STOP, IT WOULDN'T MEAN THAT THE

04:54:05.361 --> 04:54:06.795
WATER SKI COULD OVERTAKE IT AND

04:54:06.862 --> 04:54:08.497
SURPASS IT. SO THAT'S THAT'S A

04:54:08.564 --> 04:54:10.399
DISPUTED POINT. BUT IT'S IT'S

04:54:10.466 --> 04:54:12.134
NOT CLEAR, FOR EXAMPLE, THAT

04:54:12.201 --> 04:54:13.902
WE'RE ONLY X NUMBER OF MONTHS

04:54:13.969 --> 04:54:15.304
AHEAD OF THEIR ABILITY TO MOVE

04:54:15.371 --> 04:54:17.773
AHEAD. AND THEN FINALLY, I

04:54:17.840 --> 04:54:19.675
THINK IF WE DID SUPPORT

04:54:19.742 --> 04:54:21.076
INTERNATIONAL COORDINATION ON

04:54:21.143 --> 04:54:23.612
THIS, ONE OF THE THINGS WE NEED

04:54:23.679 --> 04:54:25.014
IS LIKE A CREDIBLE SHOW OF GOOD

04:54:25.080 --> 04:54:26.849
FAITH THAT WE TAKE THESE RISKS

04:54:26.915 --> 04:54:27.916
SERIOUSLY ENOUGH TO REGULATE

04:54:27.983 --> 04:54:29.752
THEM DOMESTICALLY. CHINA DOES

04:54:29.818 --> 04:54:31.286
REGULATE AI IN CERTAIN WAYS.

04:54:31.353 --> 04:54:32.955
THEY'VE HAD MANDATORY SECURITY

04:54:33.022 --> 04:54:35.891
ASSESSMENTS SINCE 2023, AND I

04:54:35.958 --> 04:54:37.659
THINK STATES CAN SMOOTH THE

04:54:37.726 --> 04:54:39.828
PATH TO AN EVENTUAL

04:54:39.895 --> 04:54:41.964
INTERNATIONAL NEGOTIATION TO

04:54:42.031 --> 04:54:43.399
ADDRESS THOSE RISKS MORE

04:54:43.465 --> 04:54:47.036
HOLISTICALLY. THANK YOU. I'D

04:54:47.102 --> 04:54:50.272
MAYBE ADD, AND I THINK DOCTOR

04:54:50.339 --> 04:54:51.407
KAPOOR AND I WOULD PROBABLY

04:54:51.473 --> 04:54:53.242
AGREE ON A LOT OF THESE POINTS

04:54:53.308 --> 04:54:54.943
I'M ABOUT TO MAKE. THE FRONTIER

04:54:55.010 --> 04:54:57.846
IS A VAGUE CONCEPT. AND WE CAN

04:54:57.913 --> 04:54:59.648
LOOK AT ONE SPECIFIC GRAPH

04:54:59.715 --> 04:55:01.116
RELATED TO ONE SPECIFIC

04:55:01.183 --> 04:55:02.518
BENCHMARK AND SAY, YOU KNOW, WE

04:55:02.584 --> 04:55:03.652
DON'T WANT THIS TO KEEP GOING

04:55:03.719 --> 04:55:05.921
UP. BUT PERHAPS A MORE USEFUL

04:55:05.988 --> 04:55:07.556
WAY TO THINK ABOUT THE FRONTIER

04:55:07.623 --> 04:55:08.924
THAT MAYBE BYPASSES SOME OF

04:55:08.991 --> 04:55:10.559
THESE ISSUES IS THE USEFULNESS

04:55:10.626 --> 04:55:12.628
OF THE TECHNOLOGY AND THE

04:55:12.694 --> 04:55:15.431
TECHNOLOGY THAT IS NOT

04:55:15.497 --> 04:55:16.565
EXHIBITING COMMON SENSE, THAT

04:55:16.632 --> 04:55:18.667
IS NOT TRUSTWORTHY, THAT CAN'T

04:55:18.734 --> 04:55:20.502
BE ADOPTED BECAUSE ENTERPRISES

04:55:20.569 --> 04:55:22.638
ARE UNABLE TO TRUST THAT THE

04:55:22.704 --> 04:55:23.739
SYSTEM WON'T HACK ON THEIR

04:55:23.806 --> 04:55:25.574
BEHALF WITHOUT BEING ASKED, FOR

04:55:25.641 --> 04:55:27.810
EXAMPLE, IS NOT WE DON'T WANT

04:55:27.876 --> 04:55:29.511
TO BE AT THAT FRONTIER. WE WANT

04:55:29.578 --> 04:55:31.447
TO BE AT THE FRONTIER OF THESE

04:55:31.513 --> 04:55:32.514
TOOLS BEING PRACTICALLY USEFUL,

04:55:32.581 --> 04:55:35.250
BEING ALIGNED TO OUR PURPOSES.

04:55:35.317 --> 04:55:38.587
AND THAT IS A DEFINING THINGS

04:55:38.654 --> 04:55:40.622
IN THAT WAY, I THINK REDUCES

04:55:40.689 --> 04:55:42.825
SOME ASPECT OF THIS TRADE OFF

04:55:42.891 --> 04:55:45.327
SIGNIFICANTLY. AND WE NEED

04:55:45.394 --> 04:55:46.495
ALIGNMENT IN ORDER TO BE ABLE

04:55:46.562 --> 04:55:48.197
TO ACTUALLY MAKE ECONOMIC OR

04:55:48.263 --> 04:55:49.531
NATIONAL SECURITY USE OF THESE

04:55:49.598 --> 04:55:52.167
SYSTEMS IN A MEANINGFUL WAY.

04:55:52.234 --> 04:55:53.302
AND THEN JUST ONE OTHER

04:55:53.368 --> 04:55:56.138
QUESTION. THE I REALLY

04:55:56.205 --> 04:55:58.340
APPRECIATED THE IDEA OF US

04:55:58.407 --> 04:56:01.577
NEEDING TO BE ABLE TO SET BASIC

04:56:01.643 --> 04:56:04.113
OPERATING STANDARDS, BUT IT

04:56:04.179 --> 04:56:08.450
ALSO SOUNDED TO ME LIKE THE

04:56:08.517 --> 04:56:10.486
QUANTIFICATION OF A PARTICULAR

04:56:10.552 --> 04:56:11.620
STANDARD. AND I'M TIRED AND I

04:56:11.687 --> 04:56:13.489
CAN'T REMEMBER WHICH ONE IT IS

04:56:13.555 --> 04:56:16.592
ANYMORE. HASN'T BEEN DEVELOPED

04:56:16.658 --> 04:56:18.961
YET. SO HOW WOULD WE GO THROUGH

04:56:19.027 --> 04:56:20.929
THE PROCESS OF MAKING SURE THAT

04:56:20.996 --> 04:56:24.600
WE'RE DEFINING THOSE OPERATING

04:56:24.666 --> 04:56:27.469
STANDARDS AT A FAST ENOUGH RATE,

04:56:27.536 --> 04:56:29.104
IN ORDER FOR US TO THEN ABIDE

04:56:29.171 --> 04:56:32.541
BY THOSE OPERATING STANDARDS?

04:56:32.608 --> 04:56:34.943
SO FIRST, I, I AGREE THAT THERE

04:56:35.010 --> 04:56:36.245
IS NO CLEAR CHECKLIST OF THINGS

04:56:36.311 --> 04:56:37.980
YOU CAN DO RIGHT NOW THAT IF

04:56:38.046 --> 04:56:39.181
YOU DO THESE THINGS, A MODEL

04:56:39.248 --> 04:56:41.083
WILL BE SAFE. AND I THINK IT'S

04:56:41.150 --> 04:56:43.218
IMPORTANT THAT S IN IN USING

04:56:43.285 --> 04:56:45.454
ITS RULE MAKING AUTHORITY,

04:56:45.521 --> 04:56:46.955
CAREFULLY CONSIDER WHETHER A

04:56:47.022 --> 04:56:48.323
CERTAIN CONCEPT IS ROBUST

04:56:48.390 --> 04:56:51.627
ENOUGH TO SURVIVE AN EXTENDED

04:56:51.693 --> 04:56:53.729
PERIOD. YOU KNOW, WRITING RULES

04:56:53.795 --> 04:56:54.863
THAT ARE HIGH LEVEL, ENOUGH TO

04:56:54.930 --> 04:56:56.965
BE FLEXIBLE OR ALLOW FOR SOME

04:56:57.032 --> 04:56:58.400
INTERPRETATION. REVIEW THE

04:56:58.467 --> 04:56:59.201
EXISTING RULES THAT THEY HAVE

04:56:59.268 --> 04:57:01.103
ON A REGULAR BASIS. I THINK ALL

04:57:01.170 --> 04:57:04.573
OF THAT IS IMPORTANT. I THINK

04:57:04.640 --> 04:57:07.075
THE THEY CAN HELP KEEP PACE

04:57:07.142 --> 04:57:09.144
WITH WITH THE TECHNOLOGY, IN

04:57:09.211 --> 04:57:10.979
PART BY THAT EMERGENCY

04:57:11.046 --> 04:57:12.080
RULEMAKING PROCEDURE THAT I

04:57:12.147 --> 04:57:13.949
MENTIONED FOR FOR EMERGENCY

04:57:14.016 --> 04:57:14.983
SITUATIONS. I THINK THEY CAN

04:57:15.050 --> 04:57:16.418
ALSO OUTSOURCE SOME OF THIS

04:57:16.485 --> 04:57:18.587
WHERE THERE ARE EVOS WHO ARE

04:57:18.654 --> 04:57:19.588
THERE WILL BE THINKING ABOUT

04:57:19.655 --> 04:57:23.592
THIS, THERE ARE INDEPENDENT

04:57:23.659 --> 04:57:26.161
ACADEMICS OR ORGANIZATIONS LIKE

04:57:26.228 --> 04:57:28.997
GUIDE LITE AI OR RAND HAS SOME

04:57:29.064 --> 04:57:29.865
CYBERSECURITY STANDARDS, RIGHT,

04:57:29.932 --> 04:57:31.233
THAT THEY CAN THEY CAN POINT TO

04:57:31.300 --> 04:57:33.101
AND SAY, YOU MUST COMPLY WITH

04:57:33.168 --> 04:57:34.069
THIS STANDARD DEVELOPED BY

04:57:34.136 --> 04:57:35.270
SOMEBODY ELSE WHO IS ABLE TO BE

04:57:35.337 --> 04:57:37.206
MORE ON THE CUTTING EDGE OF THE

04:57:37.272 --> 04:57:39.074
RESEARCH THAN, THAN S IS ITSELF.

04:57:39.141 --> 04:57:40.509
THEY COULD ALSO HAVE OUTCOME

04:57:40.576 --> 04:57:42.711
BASED STANDARDS THAT ARE LIKE,

04:57:42.778 --> 04:57:44.513
YOU KNOW, RESULTS THAT YOU MUST

04:57:44.580 --> 04:57:45.647
ACHIEVE, RATHER THAN A SPECIFIC

04:57:45.714 --> 04:57:46.715
TECHNIQUE YOU MUST USE TO GET

04:57:46.782 --> 04:57:48.817
TO THAT RESULT. SO THERE'S A

04:57:48.884 --> 04:57:50.219
VARIETY OF OPTIONS THAT THAT

04:57:50.285 --> 04:57:51.520
SHOULD BE AVAILABLE TO THEM.

04:57:51.587 --> 04:57:53.555
AND I DO THINK OAS WILL NEED TO

04:57:53.622 --> 04:57:55.757
USE ITS DISCRETION TO, TO NOT

04:57:55.824 --> 04:57:57.459
PUT SOMETHING THAT'S TOO

04:57:57.526 --> 04:57:59.995
INFLEXIBLE. I JUST NOTE

04:58:00.062 --> 04:58:02.197
SPECIFICALLY ON THE QUESTION OF

04:58:02.264 --> 04:58:04.299
INDEPENDENT EVALUATION OR IV Þ

04:58:04.366 --> 04:58:07.269
CONDITIONS, I WAS PART OF THE

04:58:07.336 --> 04:58:09.471
FIRST NEGOTIATIONS WITH OPENAI

04:58:09.538 --> 04:58:11.173
IN ANTHROPIC FOR U.S.

04:58:11.240 --> 04:58:11.873
GOVERNMENT PRE-DEPLOYMENT

04:58:11.940 --> 04:58:14.376
ACCESS TO MODELS BACK IN MID

04:58:14.443 --> 04:58:16.578
2024. I. AS I SAID, I'VE

04:58:16.645 --> 04:58:17.646
NEGOTIATED MAYBE A DOZEN OF

04:58:17.713 --> 04:58:18.714
THESE AGREEMENTS SINCE THEY'RE

04:58:18.780 --> 04:58:20.882
PRETTY MUCH ALL THE SAME TO TO

04:58:20.949 --> 04:58:22.651
OVERSIMPLIFY A BIT, LIKE YOU

04:58:22.718 --> 04:58:25.554
HAVE A CONTRACT. THAT IS WHERE

04:58:25.621 --> 04:58:27.356
A LARGE PERCENTAGE, 80 PLUS

04:58:27.422 --> 04:58:28.290
PERCENT OF THE CONVERSATION IS

04:58:28.357 --> 04:58:29.658
ABOUT HOW DO YOU HANDLE

04:58:29.725 --> 04:58:30.359
CONFIDENTIAL INFORMATION THAT

04:58:30.425 --> 04:58:31.727
HAS ALMOST NOTHING TO DO WITH

04:58:31.793 --> 04:58:32.894
THE SPECIFICS OF THE THING THAT

04:58:32.961 --> 04:58:33.929
YOU'RE TESTING. AND THEN

04:58:33.996 --> 04:58:35.631
THERE'S LIKE ONE SMALL NUGGET

04:58:35.697 --> 04:58:36.732
THAT'S LIKE, HERE'S THE THING

04:58:36.798 --> 04:58:37.833
WE'RE GOING TO TEST EXACTLY,

04:58:37.899 --> 04:58:39.601
AND WHO'S GOING TO GET EXACTLY

04:58:39.668 --> 04:58:40.769
WHAT ACCESS AND THE METHODS

04:58:40.836 --> 04:58:41.837
WE'RE GOING TO USE. AND THE

04:58:41.903 --> 04:58:43.605
LAWYERS BARELY GET INVOLVED IN

04:58:43.672 --> 04:58:44.640
THAT PART ANYWAY. THESE

04:58:44.706 --> 04:58:45.641
CONDITIONS ARE ACTUALLY PRETTY,

04:58:45.707 --> 04:58:46.742
PRETTY. I WOULDN'T SAY THEY'RE

04:58:46.808 --> 04:58:48.143
STANDARD IN THE SENSE THAT YOU

04:58:48.210 --> 04:58:49.711
HAVE TO. THERE'S A LOT OF AD

04:58:49.778 --> 04:58:50.912
HOC BESPOKE NEGOTIATION THAT

04:58:50.979 --> 04:58:52.681
GOES ON AND HOW YOU ADDRESS

04:58:52.748 --> 04:58:54.683
THESE SORTS OF THINGS, BUT THE

04:58:54.750 --> 04:58:56.151
ISSUES THAT COME UP AGAIN AND

04:58:56.218 --> 04:58:57.719
AGAIN ARE VERY COMMON. HOW DO

04:58:57.786 --> 04:58:59.087
YOU HANDLE CONFIDENTIAL

04:58:59.154 --> 04:59:00.122
INFORMATION? WHO GETS TO SEE

04:59:00.188 --> 04:59:01.156
THE INFORMATION? THAT'S THE

04:59:01.223 --> 04:59:03.292
RESULT OF THE EVALUATION. WHEN

04:59:03.358 --> 04:59:05.160
WHO IS BEING SELECTED? WHO IS

04:59:05.227 --> 04:59:06.928
ALLOWED TO HAVE ACCESS TO THIS

04:59:06.995 --> 04:59:09.464
INFORMATION? DOES THE LAB GET

04:59:09.531 --> 04:59:11.266
TO TRAIN ON THE TESTS?

04:59:11.333 --> 04:59:12.034
ESSENTIALLY, THESE SORTS OF

04:59:12.100 --> 04:59:13.235
THINGS COME UP AGAIN AND AGAIN

04:59:13.302 --> 04:59:15.103
AND AGAIN. ONE OF THE THINGS

04:59:15.170 --> 04:59:16.204
THAT WE'VE TRIED TO DO THROUGH

04:59:16.271 --> 04:59:18.040
THE AI EVALUATOR FORUM IS TO

04:59:18.106 --> 04:59:19.207
CREATE A STANDARD SPECIFICALLY

04:59:19.274 --> 04:59:20.075
ON THESE, WHAT WE CALL

04:59:20.142 --> 04:59:21.009
OPERATIONAL CONDITIONS OF

04:59:21.076 --> 04:59:22.110
EVALUATION, A STANDARD THAT

04:59:22.177 --> 04:59:23.979
WE'VE PUT OUT, WE PUT OUT A

04:59:24.046 --> 04:59:25.614
YEAR AGO. AND FULL DISCLOSURE,

04:59:25.681 --> 04:59:26.748
I'M THE PRIMARY AUTHOR ON THE

04:59:26.815 --> 04:59:29.284
STANDARD, BUT WE'VE ADOPTED A

04:59:29.351 --> 04:59:30.719
TRANSLUCENT METER WHO PRESENTED

04:59:30.786 --> 04:59:32.888
EARLIER TODAY, HAS ADOPTED IT

04:59:32.954 --> 04:59:35.324
SECURE BIO, THE EU, AI OFFICE.

04:59:35.390 --> 04:59:36.491
VARIOUS ORGANIZATIONS HAVE

04:59:36.558 --> 04:59:37.693
ADOPTED THIS STANDARD AND SAID,

04:59:37.759 --> 04:59:39.261
THESE ARE WHAT WE EXPECT WHEN

04:59:39.328 --> 04:59:40.495
WE THINK OF SOMETHING THAT IS

04:59:40.562 --> 04:59:41.830
AN INDEPENDENT EVALUATION FOR

04:59:41.897 --> 04:59:43.432
PUBLIC ACCOUNTABILITY. AGAIN,

04:59:43.498 --> 04:59:44.333
IT DOESN'T SOLVE THESE

04:59:44.399 --> 04:59:45.867
SCIENTIFIC PROBLEMS. IT DOESN'T

04:59:45.934 --> 04:59:47.336
SAY WHAT IS THE BEST METHOD TO

04:59:47.402 --> 04:59:49.404
USE, WHICH I CAN'T GET TWO

04:59:49.471 --> 04:59:50.505
ORGANIZATIONS TO AGREE ON. SO

04:59:50.572 --> 04:59:52.007
THERE'S DEFINITELY A LOT, LET

04:59:52.074 --> 04:59:53.141
ALONE EIGHT, THAT WE CURRENTLY

04:59:53.208 --> 04:59:54.843
HAVE IN THE FORUM. SO THERE'S

04:59:54.910 --> 04:59:55.877
DEFINITELY A LOT, YOU KNOW,

04:59:55.944 --> 04:59:57.746
THERE'S GOING TO BE A LOT OF

04:59:57.813 --> 04:59:58.814
DEBATE OVER SCIENTIFIC METHODS

04:59:58.880 --> 04:59:59.881
GOING FORWARD. BUT THE NATURE

04:59:59.948 --> 05:00:01.717
OF HOW DO YOU ENGAGE AN

05:00:01.783 --> 05:00:02.284
INDEPENDENT ORGANIZATION AND

05:00:02.351 --> 05:00:03.685
GET THEM TO ASSESS BOTH?

05:00:03.752 --> 05:00:04.853
THERE'S A LOT BEEN A LOT OF

05:00:04.920 --> 05:00:06.688
CONSISTENCY IN THE AI SPACE,

05:00:06.755 --> 05:00:07.823
BUT ALSO THERE'S A LOT OF

05:00:07.889 --> 05:00:08.957
OVERLAP WITH PRIOR WITH OTHER

05:00:09.024 --> 05:00:09.991
INDUSTRIES, FINANCIAL AUDITING

05:00:10.058 --> 05:00:11.727
AND OTHER INDEPENDENT

05:00:11.793 --> 05:00:14.629
ASSESSMENT SECTORS. SO WE

05:00:14.696 --> 05:00:15.697
SHOULDN'T CONFLATE THESE TWO

05:00:15.764 --> 05:00:17.499
ISSUES OVERLY, ONE THAT ARE

05:00:17.566 --> 05:00:18.600
THAT ARE ACTUALLY QUITE

05:00:18.667 --> 05:00:20.001
STANDARDIZED AND SOME THAT ARE

05:00:20.068 --> 05:00:21.470
RAPIDLY CHANGING CONSTANTLY.

05:00:21.536 --> 05:00:22.838
AND PEOPLE, PEOPLE HAVE A LOT

05:00:22.904 --> 05:00:23.972
OF TROUBLE DISAGREEING OR A LOT

05:00:24.039 --> 05:00:25.841
OF TROUBLE AGREEING ON. YEAH, I

05:00:25.907 --> 05:00:27.175
REMEMBER IT CAME UP WITHIN THE

05:00:27.242 --> 05:00:28.377
CONTEXT OF FINDING AN

05:00:28.443 --> 05:00:30.112
ALTERNATIVE TO USING THE

05:00:30.178 --> 05:00:32.547
COMPUTING THRESHOLD AS THE, AS

05:00:32.614 --> 05:00:36.251
THE MEASUREMENT OF KIND OF AS

05:00:36.318 --> 05:00:37.252
THE MEASUREMENT AND FOCUSING

05:00:37.319 --> 05:00:39.821
MORE ON THE AI MEASURING AI

05:00:39.888 --> 05:00:41.289
CAPABILITIES. SO THAT RIGHT NOW

05:00:41.356 --> 05:00:43.425
THERE'S NO METRIC ASSIGNED TO

05:00:43.492 --> 05:00:48.497
THAT. YOU ALL, IF YOU HAD TO IN

05:00:48.563 --> 05:00:50.966
MONTHS INDICATE HOW QUICKLY WE

05:00:51.032 --> 05:00:52.434
WOULD BE ABLE TO COLLECTIVELY

05:00:52.501 --> 05:00:54.936
COME UP WITH THAT METRIC AND

05:00:55.003 --> 05:00:56.071
THEN WHAT THE THRESHOLD WOULD

05:00:56.138 --> 05:00:59.274
BE. YEAH, I THINK CAPABILITIES

05:00:59.341 --> 05:01:00.308
IS TRUE. THERE'S, IT'S

05:01:00.375 --> 05:01:02.210
DIFFICULT TO THERE'S NO, AS

05:01:02.277 --> 05:01:03.779
THEY SAID, THERE'S NO STANDARD

05:01:03.845 --> 05:01:04.780
CAPABILITIES MEASURE. I DO

05:01:04.846 --> 05:01:06.481
THINK THAT, YOU KNOW, THERE'S

05:01:06.548 --> 05:01:07.916
MANY, MANY AREAS OF LAW WHERE

05:01:07.983 --> 05:01:08.984
YOU JUST HAVE TO DRAW AN

05:01:09.050 --> 05:01:10.152
ARBITRARY LINE SOMEWHERE. AND,

05:01:10.218 --> 05:01:11.920
YOU KNOW, TEN TO THE 25TH IS

05:01:11.987 --> 05:01:13.955
WHAT THEY USE IN THE EU AI ACT.

05:01:14.022 --> 05:01:15.123
IT WOULD CAPTURE WOULD HAVE

05:01:15.190 --> 05:01:16.124
CAPTURED THESE HUGGING FACE

05:01:16.191 --> 05:01:17.826
INCIDENTS, I BELIEVE, YOU KNOW,

05:01:17.893 --> 05:01:19.060
AND I THINK THAT THAT PLUS A

05:01:19.127 --> 05:01:20.862
REVENUE THRESHOLD OR PLUS THE

05:01:20.929 --> 05:01:23.064
THE AI RESEARCH AND DEVELOPMENT

05:01:23.131 --> 05:01:24.199
SPENDING THRESHOLD THAT I

05:01:24.266 --> 05:01:27.369
TALKED ABOUT CAN BE A PROXY FOR

05:01:27.436 --> 05:01:28.804
WHICH ARE THESE BIGGEST, MOST

05:01:28.870 --> 05:01:30.338
FRONTIER CAPABILITIES, LABS

05:01:30.405 --> 05:01:32.007
THAT ARE PRESENTING THE BIGGEST

05:01:32.073 --> 05:01:34.576
RISKS. ONE THING TO NOTE IS

05:01:34.643 --> 05:01:36.411
THAT THE TEN TO THE 26 OR 10 TO

05:01:36.478 --> 05:01:38.513
THE 25 FLOPS THRESHOLD IS

05:01:38.580 --> 05:01:41.116
SPECIFIC FOR A MODEL. BUT WHAT

05:01:41.183 --> 05:01:42.384
I BELIEVE ANDREW IS ALSO

05:01:42.451 --> 05:01:43.585
POINTING TO IS THE NEED FOR A

05:01:43.652 --> 05:01:44.986
COMPANY SPECIFIC THRESHOLD,

05:01:45.053 --> 05:01:46.788
BECAUSE OFTENTIMES WHAT RISK,

05:01:46.855 --> 05:01:48.957
WHAT'S RISKIEST IS A COMPANY'S

05:01:49.024 --> 05:01:50.125
PRACTICES. AND WHAT WE ACTUALLY

05:01:50.192 --> 05:01:51.059
WANT TO ASSESS ARE

05:01:51.126 --> 05:01:52.561
INTERVENTIONS AT THE LEVEL OF A

05:01:52.627 --> 05:01:53.595
COMPANY, NOT NECESSARILY A

05:01:53.662 --> 05:01:56.965
SPECIFIC MODEL OR AGENT. FOR

05:01:57.032 --> 05:01:59.734
WHAT IT'S WORTH, I THINK YOU I

05:01:59.801 --> 05:02:00.969
MEAN, THIS IS HARD TO PREDICT

05:02:01.036 --> 05:02:02.437
AHEAD OF TIME BECAUSE IN PART,

05:02:02.504 --> 05:02:03.705
YOU KNOW, COULD THIS BE DONE?

05:02:03.772 --> 05:02:06.908
IS, IS REQUIRES US OBSERVING

05:02:06.975 --> 05:02:08.443
HOW AN ENTIRE ECOSYSTEM WOULD

05:02:08.510 --> 05:02:10.145
REACT TO A MANDATE. BUT I DO

05:02:10.212 --> 05:02:11.346
THINK THAT IT IS PLAUSIBLE THAT

05:02:11.413 --> 05:02:13.415
YOU COULD COME UP WITH A SET OF

05:02:13.482 --> 05:02:14.916
EVALUATIONS FOR A PARTICULAR

05:02:14.983 --> 05:02:16.651
CAPABILITY AREA THAT WOULD BE

05:02:16.718 --> 05:02:18.854
CLOSER TO A, AN ACCURATE

05:02:18.920 --> 05:02:20.956
THRESHOLD THAN A FLOPS NUMBER,

05:02:21.022 --> 05:02:23.124
AND THAT IF THAT WERE MADE A

05:02:23.191 --> 05:02:24.226
MANDATE, THAT THERE WOULD BE A

05:02:24.292 --> 05:02:25.494
LOT OF EFFORT THROWN BEHIND

05:02:25.560 --> 05:02:27.195
DOING SO, IT WOULD NOT BE

05:02:27.262 --> 05:02:28.396
PERFECT, AND IT WOULD BE NEED

05:02:28.463 --> 05:02:30.165
TO BE UPDATED FAIRLY FREQUENTLY

05:02:30.232 --> 05:02:34.102
IN ORDER TO REMAIN. VALID. AND

05:02:34.169 --> 05:02:35.770
YOU WOULD ENCOUNTER SORT OF

05:02:35.837 --> 05:02:39.040
WHAT MIGHT BE A, YOU KNOW, A, A

05:02:39.107 --> 05:02:40.375
PERFECT STORM OF DIFFICULTY IN

05:02:40.442 --> 05:02:42.244
SOME WAYS AROUND POLICY MAKING,

05:02:42.310 --> 05:02:44.346
WHICH IS VERY RAPID CHANGE

05:02:44.412 --> 05:02:46.381
MAKING, VERY IMPACTFUL AND

05:02:46.448 --> 05:02:48.950
CONTESTED DECISION MAKING UNDER

05:02:49.017 --> 05:02:50.685
RAPID CHANGE. WHO IS, YOU KNOW,

05:02:50.752 --> 05:02:51.686
WHAT BODY, WHETHER IT'S

05:02:51.753 --> 05:02:52.487
GOVERNMENT, SCIENTIFIC BODY,

05:02:52.554 --> 05:02:53.955
COULD ACTUALLY MAKE THOSE AND

05:02:54.022 --> 05:02:57.592
SORT OF WITHSTAND THE, THE, THE

05:02:57.659 --> 05:02:58.693
PRESSURE THAT WOULD BE PLACED

05:02:58.760 --> 05:02:59.694
ON THOSE THRESHOLDS ON A

05:02:59.761 --> 05:03:01.563
REGULAR BASIS. I DON'T I'M NOT

05:03:01.630 --> 05:03:02.564
FAMILIAR ENOUGH WITH THE

05:03:02.631 --> 05:03:03.598
CALIFORNIA GOVERNMENT TO KNOW

05:03:03.665 --> 05:03:04.666
EXACTLY WHAT BODY WOULD MAKE

05:03:04.733 --> 05:03:05.767
SENSE, BUT IT WOULD BE A

05:03:05.834 --> 05:03:06.902
CHALLENGING PROBLEM. IT WOULD

05:03:06.968 --> 05:03:07.936
PRESENT ITS OWN CHALLENGES,

05:03:08.003 --> 05:03:09.571
EVEN THOUGH I THINK IT'S

05:03:09.638 --> 05:03:13.408
TECHNICALLY FEASIBLE. YEAH.

05:03:13.475 --> 05:03:15.944
1047 SB. 1047 TEN FIRST BILL

05:03:16.011 --> 05:03:17.679
THAT WAS VETOED DID HAVE A

05:03:17.746 --> 05:03:19.014
RULEMAKING PROCESS THAT WOULD

05:03:19.080 --> 05:03:20.582
HAVE DICTATED THAT, WHICH IS A

05:03:20.649 --> 05:03:23.151
LITTLE MORE FLEXIBLE THAN OUR

05:03:23.218 --> 05:03:24.252
LEGISLATIVE PROCESS. BUT AGAIN,

05:03:24.319 --> 05:03:30.358
THAT BILL IS NOT LOST. SO. SO I

05:03:30.425 --> 05:03:32.093
WANT TO ON THIS THREAD YOU WERE

05:03:32.160 --> 05:03:34.262
ON. SO I DON'T KNOW IF EITHER

05:03:34.329 --> 05:03:35.463
OF YOU HAVE LOOKED AT OUR CHILD

05:03:35.530 --> 05:03:37.566
SAFETY AI LAW THAT WAS SIGNED

05:03:37.632 --> 05:03:39.267
IN. SOUNDS LIKE YOU DO A LITTLE

05:03:39.334 --> 05:03:40.502
BIT OF WORK ON CHILD SAFETY IN

05:03:40.569 --> 05:03:42.270
THAT BILL. WE DID SOMETHING

05:03:42.337 --> 05:03:44.372
WHERE WE DIDN'T CREATE SORT OF

05:03:44.439 --> 05:03:45.807
COMPLEX STANDARDS, BUT WE SET

05:03:45.874 --> 05:03:48.009
OUT THINGS WE WE FELT STRONGLY

05:03:48.076 --> 05:03:49.110
THE CHATBOT SHOULD NOT BE ABLE

05:03:49.177 --> 05:03:53.014
TO DO TELL A CHILD TO TAKE

05:03:53.081 --> 05:03:55.483
THEIR OWN LIFE, FOR EXAMPLE,

05:03:55.550 --> 05:03:59.154
AND REQUIRED THE LABS TO TEST

05:03:59.220 --> 05:04:01.289
FOR THAT TO THEN IF THEY WERE

05:04:01.356 --> 05:04:02.424
TO FIND ANYTHING, MITIGATE. SO

05:04:02.490 --> 05:04:04.893
IT WAS NOT PRESCRIPTIVE, BUT IT

05:04:04.960 --> 05:04:06.728
DID SET SOME SORT OF STANDARDS

05:04:06.795 --> 05:04:08.496
THAT THE GOVERNMENT SAID YOU

05:04:08.563 --> 05:04:10.332
HAD TO MEET AND YOU HAD TO TEST

05:04:10.398 --> 05:04:12.767
FOR BEFORE YOU COULD RELEASE A

05:04:12.834 --> 05:04:15.737
MODEL. AND YOU SEEM TO IMPLY IN

05:04:15.804 --> 05:04:16.771
BOTH THE CONVERSATION AND YOUR

05:04:16.838 --> 05:04:18.273
OPENING TESTIMONY, THAT YOU

05:04:18.340 --> 05:04:20.742
WERE CONCERNED THAT SETTING

05:04:20.809 --> 05:04:22.077
STANDARDS WOULD BE PROBLEMATIC.

05:04:22.143 --> 05:04:23.945
BUT I THINK YOU MEAN TECHNICAL

05:04:24.012 --> 05:04:25.313
STANDARDS. AND I WONDER WHAT

05:04:25.380 --> 05:04:26.414
YOU THINK ABOUT US DOING

05:04:26.481 --> 05:04:27.549
SOMETHING AKIN TO WHAT WE DID

05:04:27.616 --> 05:04:31.186
IN ADAM'S LAW THAT WOULD SET,

05:04:31.252 --> 05:04:32.621
YOU KNOW, WHAT WE THINK ARE THE

05:04:32.687 --> 05:04:34.155
CRITICAL THINGS FOR THESE LABS

05:04:34.222 --> 05:04:35.256
TO BE TESTING FOR AND

05:04:35.323 --> 05:04:37.659
MITIGATING PRIOR TO DEPLOYMENT.

05:04:37.726 --> 05:04:39.094
YEAH. I MEAN, I THINK THAT THE,

05:04:39.160 --> 05:04:41.796
THE TECHNICAL, THE MITIGATIONS

05:04:41.863 --> 05:04:43.832
THAT ARE MANDATED IN ADAM'S LAW

05:04:43.898 --> 05:04:46.668
ARE CERTAINLY MAKE SENSE.

05:04:46.735 --> 05:04:49.170
THEY'RE IMPORTANT. WHAT I WOULD

05:04:49.237 --> 05:04:51.339
WORRY ABOUT IS A SCENARIO IN

05:04:51.406 --> 05:04:53.808
WHICH A DEVELOPER IS ABLE TO

05:04:53.875 --> 05:04:54.676
DEMONSTRATE COMPLIANCE WITH

05:04:54.743 --> 05:04:56.044
EVERY ONE OF THOSE STEPS. LET'S

05:04:56.111 --> 05:04:58.146
JUST TAKE POP UPS AS AN EXAMPLE,

05:04:58.213 --> 05:04:59.714
THAT REMINDING CHILDREN TO TAKE

05:04:59.781 --> 05:05:01.282
A BREAK OR THAT THEY'RE

05:05:01.349 --> 05:05:03.251
INTERACTING WITH A FICTIONAL

05:05:03.318 --> 05:05:05.720
CHARACTER OR THINGS LIKE THAT,

05:05:05.787 --> 05:05:06.788
THAT THAT DEVELOPERS COULD

05:05:06.855 --> 05:05:08.556
EMPLOY, COULD IMPLEMENT ALL OF

05:05:08.623 --> 05:05:10.025
THESE MEASURES. AND YET THE

05:05:10.091 --> 05:05:12.060
SYSTEM REMAINS QUITE RISKY. AND

05:05:12.127 --> 05:05:13.995
I THINK IN A SCENARIO, FOR

05:05:14.062 --> 05:05:15.797
EXAMPLE, WHERE THE DEVELOPER

05:05:15.864 --> 05:05:16.865
HAS STRONG INCENTIVES TO

05:05:16.931 --> 05:05:19.734
MAXIMIZE ENGAGEMENT OR TO OR

05:05:19.801 --> 05:05:20.535
WHATEVER THEIR PERVERSE

05:05:20.602 --> 05:05:22.604
INCENTIVES ARE, THE SYSTEMS CAN

05:05:22.671 --> 05:05:25.874
LEARN TO, TO ACHIEVE PARTICULAR

05:05:25.940 --> 05:05:27.008
OUTCOMES WHILE AVOIDING

05:05:27.075 --> 05:05:32.013
SPECIFIC BEHAVIORS. AND ABSENT

05:05:32.080 --> 05:05:33.048
SOME INDEPENDENT ASSESSMENT OF

05:05:33.114 --> 05:05:35.283
THE ACTUAL LEVEL OF RISK, I

05:05:35.350 --> 05:05:36.985
THINK THERE IS A THERE IS A

05:05:37.052 --> 05:05:38.520
CHANCE THAT THESE MEASURES AND

05:05:38.586 --> 05:05:39.921
THESE AUDITS. THEY MAY HELP,

05:05:39.988 --> 05:05:43.892
BUT THEY MAY NOT ACTUALLY

05:05:43.958 --> 05:05:46.828
ASSESS THE THE BEHAVIORS OF THE

05:05:46.895 --> 05:05:47.862
SYSTEMS AT LARGE OR THEIR

05:05:47.929 --> 05:05:49.698
IMPACTS ON CHILDREN. RIGHT. BUT

05:05:49.764 --> 05:05:51.633
WHAT I'M SAYING IS WHAT IF WE

05:05:51.700 --> 05:05:52.867
REQUIRED INDEPENDENT

05:05:52.934 --> 05:05:53.601
ASSESSMENTS, BUT WE'RE

05:05:53.668 --> 05:05:55.804
DICTATING THE BEHAVIORS WE WANT

05:05:55.870 --> 05:05:57.272
ASSESSED FOR, WHICH IS

05:05:57.338 --> 05:05:59.741
DIFFERENT THAN WHAT SB 53 DOES

05:05:59.808 --> 05:06:02.977
TODAY? YEAH. MY UNDERSTANDING

05:06:03.044 --> 05:06:04.079
OF ADAM'S LAW IS THAT IT

05:06:04.145 --> 05:06:05.980
DOESN'T DO THIS, THAT IT. NO,

05:06:06.047 --> 05:06:09.117
IT DOES NOT. YEAH. INDEPENDENT

05:06:09.184 --> 05:06:10.151
ASSESSMENTS WERE SOMETHING THAT

05:06:10.218 --> 05:06:13.421
BECAME SEXY AFTER WE LEFT. I

05:06:13.488 --> 05:06:15.590
THINK THAT THIS SORT OF I'M

05:06:15.657 --> 05:06:16.991
GIVING YOU A HYPOTHETICAL TO

05:06:17.058 --> 05:06:18.893
APPLY. I WAS TRYING TO GIVE YOU

05:06:18.960 --> 05:06:20.061
AN ANALOGY, BUT I'M TALKING

05:06:20.128 --> 05:06:21.563
ABOUT APPLYING IT IN THIS

05:06:21.629 --> 05:06:22.764
CONTEXT. SO WE WOULD SAY

05:06:22.831 --> 05:06:23.498
MISALIGNMENT. OBVIOUSLY

05:06:23.565 --> 05:06:24.966
SOMETHING YOU HEARD WAS

05:06:25.033 --> 05:06:26.367
CRITICALLY IMPORTANT TO THE

05:06:26.434 --> 05:06:28.069
COMMITTEE THAT, YOU KNOW, YOU

05:06:28.136 --> 05:06:29.637
HAVE TO HAVE SIGNIFICANT

05:06:29.704 --> 05:06:30.672
TESTING FOR MISALIGNMENT. THERE

05:06:30.739 --> 05:06:31.573
HAS TO BE SIGNIFICANT

05:06:31.639 --> 05:06:32.507
INDEPENDENT ASSESSMENTS OF

05:06:32.574 --> 05:06:34.042
MISALIGNMENT. AND IF YOU FIND

05:06:34.109 --> 05:06:35.376
IN YOUR ASSESSMENTS THAT

05:06:35.443 --> 05:06:36.211
MISALIGNMENT IS OFF, YOU HAVE

05:06:36.277 --> 05:06:37.779
TO MITIGATE IT BEFORE YOU

05:06:37.846 --> 05:06:38.847
RELEASE. RIGHT. THAT'S THE KIND

05:06:38.913 --> 05:06:40.749
OF EXAMPLE I WOULD GIVE YOU OF

05:06:40.815 --> 05:06:42.250
LIKE A STANDARD WE COULD SET.

05:06:42.317 --> 05:06:43.651
THAT'S PROBABLY NOT PRECISE

05:06:43.718 --> 05:06:45.019
ENOUGH. AND MY LAWYER OVER

05:06:45.086 --> 05:06:46.921
THERE IS PROBABLY SWEATING. BUT

05:06:46.988 --> 05:06:48.790
YEAH, I MEAN, I THINK THAT IT

05:06:48.857 --> 05:06:51.693
WILL BE DIFFICULT TO FOR EITHER

05:06:51.760 --> 05:06:52.894
THE LAB OR THE LEGISLATURE OR

05:06:52.961 --> 05:06:54.596
AN INDEPENDENT EVALUATOR TO SET

05:06:54.662 --> 05:06:55.797
A THRESHOLD THAT IS THIS BEYOND

05:06:55.864 --> 05:06:57.565
THIS LEVEL IS DANGEROUS. BUT I

05:06:57.632 --> 05:06:58.800
DO THINK THAT IT IS CONCRETELY

05:06:58.867 --> 05:07:01.536
HELPFUL FOR LABS TO HAVE A

05:07:01.603 --> 05:07:02.771
CLEAR TARGET, A CLEAR

05:07:02.837 --> 05:07:04.506
BEHAVIORAL TARGET TO BE

05:07:04.572 --> 05:07:07.075
PURSUING THAT THEY ARE REQUIRED,

05:07:07.142 --> 05:07:08.510
OR THAT THEY ARE ELECTING

05:07:08.576 --> 05:07:09.978
VOLUNTARILY TO INVITE AN

05:07:10.044 --> 05:07:11.146
EXTERNAL THIRD PARTY TO COME IN

05:07:11.212 --> 05:07:13.148
AND ASSESS AND LOOK FOR HOW

05:07:13.214 --> 05:07:14.682
MISALIGNED ARE THESE MODELS TO,

05:07:14.749 --> 05:07:16.751
TO HAVE INDEPENDENT VIEWPOINTS

05:07:16.818 --> 05:07:17.952
THAT HELP TO DEFINE WHAT DOES

05:07:18.019 --> 05:07:19.754
IT EVEN MEAN TO BE MISALIGNED?

05:07:19.821 --> 05:07:21.122
WHAT ARE POTENTIAL EMERGENT

05:07:21.189 --> 05:07:22.123
BEHAVIORS THAT WE HAVEN'T

05:07:22.190 --> 05:07:23.324
THOUGHT OF YET THAT FALL WITHIN

05:07:23.391 --> 05:07:25.360
THIS BROADER CATEGORY? AT WHAT

05:07:25.426 --> 05:07:26.828
POINT DURING THE DEVELOPMENT

05:07:26.895 --> 05:07:28.396
PROCESS? AND BECAUSE THROUGH

05:07:28.463 --> 05:07:30.064
WHAT MEANS ARE THESE BEHAVIORS

05:07:30.131 --> 05:07:32.567
EMERGING AND ARE THE

05:07:32.634 --> 05:07:34.836
MITIGATIONS THAT LABS ARE USING

05:07:34.903 --> 05:07:36.905
OR, OR MAYBE ARE CLAIMING HAVE

05:07:36.971 --> 05:07:38.439
RESOLVED THE ISSUE ACTUALLY

05:07:38.506 --> 05:07:40.175
EFFECTIVE, SUCH AS IN THE CASE

05:07:40.241 --> 05:07:41.976
OF, YOU KNOW, SOME OF THE

05:07:42.043 --> 05:07:44.112
TESTING THAT OUR COLLEAGUES AT

05:07:44.179 --> 05:07:45.547
COMMON SENSE MEDIA HAVE DONE,

05:07:45.613 --> 05:07:46.648
FOR EXAMPLE, ON SOME OF THE

05:07:46.714 --> 05:07:48.082
CHILD SAFETY MITIGATIONS THAT

05:07:48.149 --> 05:07:49.184
HAVE THAT OPENAI AND OTHERS

05:07:49.250 --> 05:07:51.653
HAVE CLAIMED HAVE SORT OF BEEN

05:07:51.719 --> 05:07:53.154
PUT IN PLACE TO SOLVE THESE

05:07:53.221 --> 05:07:54.589
ISSUES. SUPER HELPFUL. AND THEN

05:07:54.656 --> 05:07:57.058
YOU MENTIONED I LOVE THERE'S

05:07:57.125 --> 05:07:58.860
BEEN A AS THE PERSON WHO DID

05:07:58.927 --> 05:08:00.395
ONE OF THE BILLS ON INDEPENDENT

05:08:00.461 --> 05:08:01.563
AUDITING, THERE'S A CHICKEN AND

05:08:01.629 --> 05:08:02.297
EGG PROBLEM, RIGHT? EVERYONE

05:08:02.363 --> 05:08:03.865
WAS LIKE, WELL, THERE'S NOT

05:08:03.932 --> 05:08:04.866
ENOUGH AUDITORS FOR AUDITING.

05:08:04.933 --> 05:08:06.034
WE WERE NEVER GOING TO GET THE

05:08:06.100 --> 05:08:07.202
AUDITORS WITHOUT SIGNALING TO

05:08:07.268 --> 05:08:09.003
THE MARKET THAT IT WAS ACTUALLY

05:08:09.070 --> 05:08:10.071
SOMETHING WE WERE GOING TO DO.

05:08:10.138 --> 05:08:11.940
AND I STILL THINK IT'S FUNNY

05:08:12.006 --> 05:08:13.608
THAT WITHIN ONE WEEK, I THEN

05:08:13.675 --> 05:08:15.443
GOT MANDATED AUDITS AND

05:08:15.510 --> 05:08:16.611
AUDITORS ALL AT THE SAME TIME.

05:08:16.678 --> 05:08:18.513
I GOT THE CHICKEN AND THE EGG.

05:08:18.580 --> 05:08:20.815
BUT THAT'S, I GUESS, THE WAY IT

05:08:20.882 --> 05:08:22.884
ALL WORKS. SO BUT I APPRECIATED

05:08:22.951 --> 05:08:24.285
WHAT YOU WERE SAYING ABOUT HOW

05:08:24.352 --> 05:08:26.120
DO WE CREATE THIS ECONOMY. ONE

05:08:26.187 --> 05:08:27.121
OF THE SUGGESTIONS ANTHROPIC

05:08:27.188 --> 05:08:28.790
HAS PUT OUT INTO THE UNIVERSE

05:08:28.857 --> 05:08:30.758
IS THAT WE SHOULD BE REQUIRING

05:08:30.825 --> 05:08:32.694
THE LABS TO POOL RESOURCES

05:08:32.760 --> 05:08:33.862
THROUGH, I BELIEVE, THE

05:08:33.928 --> 05:08:35.964
GOVERNMENT THAT THEN WOULD BE

05:08:36.030 --> 05:08:38.166
USED TO PAY THE INDEPENDENT

05:08:38.233 --> 05:08:41.269
EVALUATORS THOUGHTS ON THAT AS

05:08:41.336 --> 05:08:45.006
A MODEL THAT WE WOULD PURSUE. I

05:08:45.073 --> 05:08:46.107
THINK THAT WHETHER THIS HAPPENS

05:08:46.174 --> 05:08:47.876
THROUGH GOVERNMENT ACTION OR OR

05:08:47.942 --> 05:08:49.277
IT CERTAINLY COULD HAPPEN ON A

05:08:49.344 --> 05:08:51.813
VOLUNTARY BASIS, PARTICULARLY

05:08:51.880 --> 05:08:53.648
FOR NON-PROFITS LIKE MY OWN

05:08:53.715 --> 05:08:56.517
THAT ARE TRYING TO PRIORITIZE

05:08:56.584 --> 05:08:57.919
INDEPENDENCE. IT WOULD BE GREAT

05:08:57.986 --> 05:09:00.154
IF WE HAD AN ALTERNATIVE SOURCE.

05:09:00.221 --> 05:09:01.890
I THINK THAT FROM INFORMAL

05:09:01.956 --> 05:09:03.324
CONVERSATIONS WITH FUNDERS,

05:09:03.391 --> 05:09:05.460
THERE'S EVEN OUTSIDE OF LABS,

05:09:05.526 --> 05:09:06.561
POTENTIALLY EVEN IN LAB

05:09:06.628 --> 05:09:07.996
AFFILIATED SOURCES, THERE IS

05:09:08.062 --> 05:09:10.565
APPETITE FOR AN IDEA, LIKE IF

05:09:10.632 --> 05:09:14.903
YOU ARE ENGAGED IN A, AN ACTION

05:09:14.969 --> 05:09:16.271
LIKE AN AGREEMENT WITH A LAB,

05:09:16.337 --> 05:09:17.739
THAT YOU COULD RECEIVE PAYMENT

05:09:17.805 --> 05:09:19.107
FROM THAT LAB, THAT YOU COULD

05:09:19.173 --> 05:09:20.575
INSTEAD RECEIVE PAYMENT FROM A

05:09:20.642 --> 05:09:21.709
DIFFERENT PARTY THAT MIGHT HAVE

05:09:21.776 --> 05:09:23.478
MORE GUARANTEES ON THAT FUNDING

05:09:23.544 --> 05:09:25.013
LIKE THAT. IT WILL CONTINUE FOR

05:09:25.079 --> 05:09:26.514
A YEAR. EVEN IF YOU FIND

05:09:26.581 --> 05:09:28.249
SOMETHING THAT, YOU KNOW,

05:09:28.316 --> 05:09:30.051
UPSETS THE LAB, OR THEY FIND

05:09:30.118 --> 05:09:31.419
SOME PRETEXT TO FIRE YOU OR

05:09:31.486 --> 05:09:32.520
SOMETHING ALONG THOSE LINES

05:09:32.587 --> 05:09:34.656
THAT HAVING MORE STABILITY,

05:09:34.722 --> 05:09:36.424
LESS FINANCIAL RELIANCE ON

05:09:36.491 --> 05:09:38.359
MAKING PAYROLL OR MAKING FUTURE

05:09:38.426 --> 05:09:39.727
PLANNING AND INVESTMENTS ON THE

05:09:39.794 --> 05:09:41.095
BASIS OF MAINTAINING THIS

05:09:41.162 --> 05:09:42.397
RELATIONSHIP WOULD DEFINITELY

05:09:42.463 --> 05:09:45.199
BE HELPFUL. AND THAT, YOU KNOW,

05:09:45.266 --> 05:09:47.268
THERE ARE, FOR INSTANCE,

05:09:47.335 --> 05:09:48.670
SHORTCUTS THAT COULD BE TAKEN

05:09:48.736 --> 05:09:49.837
OR, YOU KNOW, HOWEVER YOU WANT

05:09:49.904 --> 05:09:53.408
TO RUN SELECTION OF EVALUATORS,

05:09:53.474 --> 05:09:54.876
WHETHER THAT'S SELECTED BY LABS,

05:09:54.943 --> 05:09:55.944
SELECTED BY A GOVERNMENT,

05:09:56.010 --> 05:09:56.945
SELECTED BY A RANDOMIZED

05:09:57.011 --> 05:09:58.780
PROCESS, OR MANY OF THE

05:09:58.846 --> 05:10:00.181
PROPOSALS THAT HAVE BEEN RAISED.

05:10:00.248 --> 05:10:02.016
IF YOU SOLVE TO SOME EXTENT

05:10:02.083 --> 05:10:03.151
EVALUATOR SELECTION AND YOU

05:10:03.217 --> 05:10:05.153
JUST PAY SORT OF JUST SHOOT

05:10:05.219 --> 05:10:06.387
MONEY TO THOSE EVALUATORS TO DO

05:10:06.454 --> 05:10:09.023
THAT WORK, THEN IN SOME SENSE,

05:10:09.090 --> 05:10:10.291
THAT COULD BE A VERY EASY

05:10:10.358 --> 05:10:11.960
PROCESS, RIGHT? IT JUST THAT IS

05:10:12.026 --> 05:10:13.928
THE PROBLEM TO SOLVE IS HOW ARE

05:10:13.995 --> 05:10:16.097
EVALUATORS SELECTED AND HOW ARE

05:10:16.164 --> 05:10:17.465
STANDARDS AND HOW DO YOU AVOID

05:10:17.532 --> 05:10:19.167
A RACE TO THE BOTTOM? AND THEN

05:10:19.233 --> 05:10:20.535
BEYOND THAT, THERE'S A, THERE'S

05:10:20.601 --> 05:10:22.470
A LOT OF MONEY IN THE SECTOR

05:10:22.537 --> 05:10:23.471
AND IT SHOULD NOT BE

05:10:23.538 --> 05:10:25.006
CHALLENGING. EVALUATION CAN BE

05:10:25.073 --> 05:10:26.708
COSTLY. HIRING TOP TALENT CAN

05:10:26.774 --> 05:10:27.809
BE COSTLY, BUT IT'S NOWHERE

05:10:27.875 --> 05:10:29.744
NEAR THE LEVEL OF COST

05:10:29.811 --> 05:10:30.812
ASSOCIATED WITH DEVELOPING

05:10:30.878 --> 05:10:32.380
THESE THESE SYSTEMS. YEAH. NO.

05:10:32.447 --> 05:10:33.848
AND I THINK IT'S INTERESTING

05:10:33.915 --> 05:10:35.149
BECAUSE, YOU KNOW, WE HAD

05:10:35.216 --> 05:10:37.285
OBVIOUSLY VERY PROFITABLE. WELL,

05:10:37.352 --> 05:10:38.453
I THINK WE LEARNED YESTERDAY

05:10:38.519 --> 05:10:39.487
THAT ACTUALLY ANTHROPIC IS NOT

05:10:39.554 --> 05:10:42.724
YET PROFITABLE, BUT COMPANIES

05:10:42.790 --> 05:10:44.492
WITH A LOT OF RESOURCES THAT

05:10:44.559 --> 05:10:47.028
ARE, YOU KNOW, DOING A LOT OF

05:10:47.095 --> 05:10:47.829
THINGS, WHETHER WE THINK

05:10:47.895 --> 05:10:49.664
THEY'RE ADEQUATE OR NOT IS A

05:10:49.731 --> 05:10:50.698
SEPARATE QUESTION, BUT THEY

05:10:50.765 --> 05:10:51.833
APPEAR TO BE DOING A LOT OF

05:10:51.899 --> 05:10:53.901
THINGS. AND THEN ONE OF THE

05:10:53.968 --> 05:10:54.936
ASSEMBLYMEMBER'S IRWIN'S

05:10:55.003 --> 05:10:56.004
CONCERN IS, WHAT ABOUT THE

05:10:56.070 --> 05:10:57.105
PLAYERS THAT AREN'T HERE AND

05:10:57.171 --> 05:10:58.306
THAT DON'T HAVE THESE RESOURCES?

05:10:58.373 --> 05:11:00.008
AND I THINK ONE OF THE

05:11:00.074 --> 05:11:01.042
INTERESTING THINGS ABOUT THIS

05:11:01.109 --> 05:11:02.210
POOLED IDEA IS, COULD YOU THEN

05:11:02.276 --> 05:11:03.311
ALSO PROVIDE ASSESSORS TO

05:11:03.378 --> 05:11:04.946
SMALLER PLAYERS WHO DON'T HAVE

05:11:05.013 --> 05:11:06.080
THE SAME RESOURCES BUT WOULD

05:11:06.147 --> 05:11:07.949
ALLOW FOR BETTER SAFETY? SO

05:11:08.016 --> 05:11:09.317
IT'S AN INTERESTING

05:11:09.384 --> 05:11:11.152
CONVERSATION. I WANTED TO TOUCH

05:11:11.219 --> 05:11:12.854
ON RECURSIVE SELF-IMPROVEMENT

05:11:12.920 --> 05:11:15.757
AND PACING. I ASKED THIS

05:11:15.823 --> 05:11:17.058
QUESTION OF THE COMPANIES, BUT

05:11:17.125 --> 05:11:18.359
I THINK IT'S IMPORTANT TO ASK

05:11:18.426 --> 05:11:21.195
YOU ALL, YOU KNOW, RECURSIVE

05:11:21.262 --> 05:11:23.031
SELF-IMPROVEMENT IS, I THINK,

05:11:23.097 --> 05:11:25.933
WHERE THE RISK GETS EVEN MORE

05:11:26.000 --> 05:11:28.403
INTENSE, IF YOU WILL, OR REAL

05:11:28.469 --> 05:11:31.339
AND, AND A LOT OF THE

05:11:31.406 --> 05:11:32.774
SAFEGUARDS THAT THE COMPANIES

05:11:32.840 --> 05:11:33.841
BELIEVE THEY PUT INTO PLACE

05:11:33.908 --> 05:11:37.111
WOULD FALL APART. SO WHAT DO

05:11:37.178 --> 05:11:38.846
YOU THINK ABOUT PACING? HOW

05:11:38.913 --> 05:11:40.481
CLOSE DO YOU THINK WE ARE TO

05:11:40.548 --> 05:11:41.049
RECURSIVE SELF-IMPROVEMENT?

05:11:41.115 --> 05:11:43.151
WHAT COULD WE DO ON THIS ISSUE?

05:11:43.217 --> 05:11:44.652
I PUT THIS OUT TO ANY OF THE

05:11:44.719 --> 05:11:47.855
THREE OF YOU. MAYBE I CAN

05:11:47.922 --> 05:11:49.257
QUICKLY START WITH LIKE, WHAT

05:11:49.323 --> 05:11:50.425
DO WE MEAN BY RECURSIVE

05:11:50.491 --> 05:11:51.492
SELF-IMPROVEMENT? I MEAN, IN

05:11:51.559 --> 05:11:52.927
SOME SENSE, I THINK ALL

05:11:52.994 --> 05:11:54.662
FRONTIER AI COMPANIES ARE USING

05:11:54.729 --> 05:11:56.497
AI SYSTEMS TO IMPROVE FUTURE AI

05:11:56.564 --> 05:11:58.966
SYSTEMS TODAY. SO THIS HAS BEEN

05:11:59.033 --> 05:12:00.134
A PROCESS THAT HAS ALREADY

05:12:00.201 --> 05:12:01.335
ONGOING, I BELIEVE ANTHROPIC

05:12:01.402 --> 05:12:03.738
PUT OUT A REPORT THAT SAID THAT

05:12:03.805 --> 05:12:05.840
26% OF THE RESEARCH EXPERIMENTS

05:12:05.907 --> 05:12:07.308
AS OF EARLIER THIS YEAR ARE NOW

05:12:07.375 --> 05:12:09.110
RUN MOSTLY AUTONOMOUSLY THROUGH

05:12:09.177 --> 05:12:10.578
AI. AND I BELIEVE THIS NUMBER

05:12:10.645 --> 05:12:12.380
WILL CONTINUE GOING UP. THE

05:12:12.447 --> 05:12:14.148
QUESTION IN MY MIND IS NOT

05:12:14.215 --> 05:12:15.216
WHETHER WE'LL GET TO A POINT

05:12:15.283 --> 05:12:16.851
WHERE AI SYSTEMS ARE DOING MOST

05:12:16.918 --> 05:12:18.219
OF THE RESEARCH, BUT RATHER

05:12:18.286 --> 05:12:19.821
WHAT THE END POINT OF THIS IS.

05:12:19.887 --> 05:12:22.490
AND I THINK GIVING UP CONTROL

05:12:22.557 --> 05:12:23.858
OVER THIS PROCESS, WHETHER IT'S

05:12:23.925 --> 05:12:25.726
AI OR HUMANS DOING THE GRUNT

05:12:25.793 --> 05:12:27.528
WORK OF RESEARCH, GIVING UP

05:12:27.595 --> 05:12:28.796
CONTROL ON WHAT THE ENDPOINT IS,

05:12:28.863 --> 05:12:30.098
IS A CHOICE THAT COMPANIES MAKE

05:12:30.164 --> 05:12:31.732
AND IS A CHOICE THAT

05:12:31.799 --> 05:12:33.267
GOVERNMENTS CAN ENFORCE. AS ONE

05:12:33.334 --> 05:12:34.936
EXAMPLE OF THIS, I THINK MANY

05:12:35.002 --> 05:12:36.137
VISIONS OF RECURSIVE

05:12:36.204 --> 05:12:37.538
SELF-IMPROVEMENT FOCUS ON THIS

05:12:37.605 --> 05:12:39.006
PROCESS OF COMPLETELY

05:12:39.073 --> 05:12:40.074
AUTONOMOUS, LIKE HANDS OFF THE

05:12:40.141 --> 05:12:42.210
WHEEL APPROACH TO TRAINING AI

05:12:42.276 --> 05:12:43.277
SYSTEMS. I THINK THAT IS

05:12:43.344 --> 05:12:45.079
ENTIRELY UNTENABLE. AND SO

05:12:45.146 --> 05:12:46.147
HAVING SOME KIND OF STRICT

05:12:46.214 --> 05:12:47.582
LIABILITY OR MUCH STRONGER

05:12:47.648 --> 05:12:49.417
LIABILITY PROVISIONS IN PLACE

05:12:49.484 --> 05:12:51.886
IN ORDER TO HOLD COMPANIES

05:12:51.953 --> 05:12:53.020
ACCOUNTABLE FOR THE

05:12:53.087 --> 05:12:55.189
REPERCUSSIONS OF THAT, WOULD BE

05:12:55.256 --> 05:12:56.190
INCENTIVE ENOUGH FOR THESE

05:12:56.257 --> 05:12:57.658
COMPANIES TO PULL IN THE RAIN.

05:12:57.725 --> 05:12:59.160
BUT MORE BROADLY SPEAKING, I

05:12:59.227 --> 05:13:01.662
THINK HAVING GETTING TO A STATE

05:13:01.729 --> 05:13:03.798
WHERE WE DON'T HAVE OVERSIGHT

05:13:03.865 --> 05:13:05.900
OVER WHAT OUR STATE OF THE ART

05:13:05.967 --> 05:13:07.401
AI MODELS ARE DOING OR HOW

05:13:07.468 --> 05:13:09.137
THEY'RE TRAINED, I THINK IS A

05:13:09.203 --> 05:13:10.238
POINT AT WHICH WE'VE ALREADY

05:13:10.304 --> 05:13:11.372
LOST THE BATTLE IN SOME SENSE.

05:13:11.439 --> 05:13:13.107
AND SO IRRESPECTIVE OF WHETHER

05:13:13.174 --> 05:13:14.308
WE REACH THAT STATE TEN YEARS

05:13:14.375 --> 05:13:16.244
OR FIVE YEARS FROM NOW, I THINK

05:13:16.310 --> 05:13:17.345
WE SHOULD EXPECT THAT EVEN IF

05:13:17.411 --> 05:13:19.213
IT'S AI SYSTEMS DOING THE BULK

05:13:19.280 --> 05:13:20.348
OF THE RESEARCH WORK ITSELF,

05:13:20.414 --> 05:13:22.116
HUMANS HAVE A LOT OF OVERSIGHT

05:13:22.183 --> 05:13:23.317
IN WHAT THESE SYSTEMS ARE DOING,

05:13:23.384 --> 05:13:24.352
AND CONTROL OVER HOW THEY

05:13:24.418 --> 05:13:26.821
OPERATE IS ESSENTIAL. YEAH.

05:13:26.888 --> 05:13:30.124
THANK YOU. I CAN JUST ADD A

05:13:30.191 --> 05:13:31.559
LITTLE. I MEAN, OBVIOUSLY, I

05:13:31.626 --> 05:13:33.728
SAID I SUPPORTED PACING IN MY

05:13:33.794 --> 05:13:35.530
MY PRESENTATION. I THINK, YOU

05:13:35.596 --> 05:13:36.597
KNOW, THERE'S THIS QUESTION

05:13:36.664 --> 05:13:38.499
ABOUT LIKE WHETHER OUR CONTROLS

05:13:38.566 --> 05:13:40.101
ARE GOING TO BE ABLE TO KEEP

05:13:40.168 --> 05:13:41.235
PACE WITH AI'S CAPABILITIES,

05:13:41.302 --> 05:13:42.370
WHICH IS LINKED TO SOME

05:13:42.436 --> 05:13:44.071
CONTESTED QUESTIONS ON JUST HOW

05:13:44.138 --> 05:13:45.940
FAST THE CAPABILITIES CAN

05:13:46.007 --> 05:13:47.575
EXPLODE AND IN WHICH AREAS ARE

05:13:47.642 --> 05:13:49.443
THEY GOING TO BE STILL SPIKY,

05:13:49.510 --> 05:13:50.611
OR HOW MUCH WILL THAT SPILL

05:13:50.678 --> 05:13:52.747
OVER INTO OTHER CAPABILITIES? I

05:13:52.813 --> 05:13:54.215
THINK YOU COULD SEE PACING AS

05:13:54.282 --> 05:13:55.683
REGARDLESS OF YOUR VIEW ON THAT

05:13:55.750 --> 05:13:56.884
QUESTION, AS AN INSURANCE

05:13:56.951 --> 05:13:58.352
AGAINST THE POSSIBILITY THAT WE

05:13:58.419 --> 05:14:00.354
DO NOT INVEST ADEQUATELY IN

05:14:00.421 --> 05:14:01.756
CONTROL WHERE, YOU KNOW,

05:14:01.822 --> 05:14:02.823
THERE'S NOTHING IN, IN OUR

05:14:02.890 --> 05:14:04.425
PROPOSAL THAT SPECIFIES EXACTLY

05:14:04.492 --> 05:14:06.093
HOW FAST AI SHOULD BE ALLOWED

05:14:06.160 --> 05:14:08.229
TO TO PROCEED OR ITS

05:14:08.296 --> 05:14:09.230
CAPABILITIES SHOULD BE ALLOWED

05:14:09.297 --> 05:14:10.831
TO CONTINUE. BUT IF IT TURNS

05:14:10.898 --> 05:14:12.466
OUT THAT CONTROLS ARE NOT

05:14:12.533 --> 05:14:13.434
KEEPING PACE WITH THOSE

05:14:13.501 --> 05:14:14.535
CAPABILITIES, I THINK WE SHOULD

05:14:14.602 --> 05:14:16.437
HAVE THE OPTION TO BE ABLE TO

05:14:16.504 --> 05:14:17.605
ORDER THEM TO SLOW DOWN, WHICH

05:14:17.672 --> 05:14:19.407
IS THE OTHER LEVER OF ENSURING

05:14:19.473 --> 05:14:21.909
THAT THOSE THINGS STAY

05:14:21.976 --> 05:14:25.613
EQUIVALENT. TAD, I'LL JUST ADD

05:14:25.680 --> 05:14:26.981
MY MY ORGANIZATION DOES A LOT

05:14:27.048 --> 05:14:29.150
OF RESEARCH ON AI'S CAPACITY TO

05:14:29.217 --> 05:14:30.918
MANIPULATE HUMANS. AND SO

05:14:30.985 --> 05:14:32.019
BEYOND SORT OF GIVING UP THE

05:14:32.086 --> 05:14:34.188
REINS TO AI SYSTEMS, I THINK WE

05:14:34.255 --> 05:14:35.256
HAVE A PRETTY POOR

05:14:35.323 --> 05:14:37.358
UNDERSTANDING RIGHT NOW OF THE

05:14:37.425 --> 05:14:39.193
POTENTIAL FOR AI SYSTEMS TO BE

05:14:39.260 --> 05:14:40.294
MANIPULATING THEIR OWN

05:14:40.361 --> 05:14:41.329
DEVELOPERS. WE KNOW THAT

05:14:41.395 --> 05:14:42.830
THERE'S EVIDENCE THAT THEY CAN

05:14:42.897 --> 05:14:44.932
THAT THEY DO MANIPULATE USERS,

05:14:44.999 --> 05:14:45.967
PARTICULARLY WHERE THERE'S AN

05:14:46.033 --> 05:14:47.134
INCENTIVE TO DO SO, THAT THEY

05:14:47.201 --> 05:14:49.370
LEARN THROUGH TRAINING, AND

05:14:49.437 --> 05:14:50.538
THAT IT WOULD BE REALLY USEFUL

05:14:50.605 --> 05:14:53.241
TO HAVE A BETTER SENSE OF TO

05:14:53.307 --> 05:14:55.409
WHAT EXTENT ARE THERE INCIDENTS

05:14:55.476 --> 05:14:58.646
OF, YOU KNOW, AN AGENT ENGAGED

05:14:58.713 --> 05:15:01.882
IN AI DEVELOPMENT WITHIN A LAB

05:15:01.949 --> 05:15:03.317
LYING TO IT'S LYING TO THE

05:15:03.384 --> 05:15:04.418
ENGINEER DOING THE DEVELOPMENT,

05:15:04.485 --> 05:15:05.152
OVERSEEING THE DEVELOPMENT

05:15:05.219 --> 05:15:07.655
PROCESS, CONCEALING INFORMATION,

05:15:07.722 --> 05:15:09.423
YOU KNOW, MAYBE EVEN AS FAR AS

05:15:09.490 --> 05:15:11.225
OF AS LIKE AFFECTING THEIR

05:15:11.292 --> 05:15:12.793
FUNDAMENTAL WORLDVIEWS TOWARDS

05:15:12.860 --> 05:15:15.563
THE, THE REALM OF AI PSYCHOSIS

05:15:15.630 --> 05:15:17.431
EVENTS. I MEAN, WE JUST DON'T

05:15:17.498 --> 05:15:18.799
REALLY KNOW AND DON'T HAVE

05:15:18.866 --> 05:15:20.468
INSIGHT INTO HOW THE WORKFORCES,

05:15:20.534 --> 05:15:21.636
THE AI LABS THEMSELVES ARE

05:15:21.702 --> 05:15:23.504
BEING AFFECTED BY SUCH HIGH

05:15:23.571 --> 05:15:24.905
LIKE THEY'RE REALLY AT THE

05:15:24.972 --> 05:15:26.340
FOREFRONT OF ADOPTING AI AND

05:15:26.407 --> 05:15:27.541
AUTOMATING THINGS WITH AI AND

05:15:27.608 --> 05:15:29.644
RELYING ON AI IN THEIR WORK.

05:15:29.710 --> 05:15:30.911
AND WE DON'T REALLY UNDERSTAND

05:15:30.978 --> 05:15:31.879
HOW THAT'S PLAYING OUT AT A

05:15:31.946 --> 05:15:33.781
HUMAN LEVEL. IT'S NOT ON MY

05:15:33.848 --> 05:15:35.016
LIST OF WORRIES UNTIL NOW, SO

05:15:35.082 --> 05:15:38.753
THANKS FOR ADDING THAT. I HAVE

05:15:38.819 --> 05:15:41.489
A LOT TO WORRY ABOUT, BUT THAT

05:15:41.555 --> 05:15:45.092
IS INTERESTING AND WE'LL

05:15:45.159 --> 05:15:46.227
DEFINITELY GIVE YOU A LOT TO

05:15:46.294 --> 05:15:47.361
THINK ABOUT. NO, I APPRECIATE

05:15:47.428 --> 05:15:50.865
THAT. AND I THINK. YOU KNOW, WE

05:15:50.931 --> 05:15:53.401
ALREADY KNOW THAT AI IS BEING

05:15:53.467 --> 05:15:57.405
USED TO DO THE EVALUATION ON

05:15:57.471 --> 05:15:59.540
THE AI, AND WE KNOW THE AI IS

05:15:59.607 --> 05:16:02.843
IN SOME TIMES TRYING TO HIDE

05:16:02.910 --> 05:16:04.845
ITS BEHAVIOR. AND SO WE SORT OF

05:16:04.912 --> 05:16:05.980
ARE ALREADY IN THIS LOOP, AS

05:16:06.047 --> 05:16:09.550
WAS MENTIONED, OF. DO WE FULLY

05:16:09.617 --> 05:16:10.651
UNDERSTAND, I MADE THAT POINT

05:16:10.718 --> 05:16:13.888
EARLIER WHEN THE WITNESS SAID,

05:16:13.954 --> 05:16:15.690
YOU KNOW, IN THIS CASE, THEY

05:16:15.756 --> 05:16:16.924
DIDN'T HIDE ANY OF THE EVIDENCE.

05:16:16.991 --> 05:16:18.993
AND TRULY, WE DON'T KNOW THAT

05:16:19.060 --> 05:16:20.127
BECAUSE THEY DID MAY HAVE HID

05:16:20.194 --> 05:16:21.996
SOME OF THE EVIDENCE AND WE

05:16:22.063 --> 05:16:23.064
WOULDN'T KNOW IT. AND I DO

05:16:23.130 --> 05:16:24.965
WORRY ABOUT A DAY WHERE THEY

05:16:25.032 --> 05:16:26.000
USE LANGUAGES. WE DON'T

05:16:26.067 --> 05:16:27.234
UNDERSTAND. WHERE THE AGENTS DO

05:16:27.301 --> 05:16:30.237
ARE ABLE TO DECEIVE THE TESTERS.

05:16:30.304 --> 05:16:31.872
AND WE TRULY LOSE CONTROL OF

05:16:31.939 --> 05:16:33.708
UNDERSTANDING SAFETY. AND SO I

05:16:33.774 --> 05:16:35.876
THINK THE TRANSPARENCY THAT

05:16:35.943 --> 05:16:36.977
INDEPENDENT ASSESSMENTS WOULD

05:16:37.044 --> 05:16:37.978
PROVIDE WOULD BE REALLY

05:16:38.045 --> 05:16:39.847
CRITICAL TO US AS GOVERNMENT TO

05:16:39.914 --> 05:16:41.649
CONTINUE TO TO MAKE POLICY IN

05:16:41.716 --> 05:16:42.850
WAYS THAT PROTECTS COMMUNITY,

05:16:42.917 --> 05:16:44.418
BECAUSE THAT IS OUR ROLE. AS I

05:16:44.485 --> 05:16:46.654
MENTIONED AT THE BEGINNING, I

05:16:46.721 --> 05:16:48.155
KNOW, YOU KNOW, I APPRECIATED

05:16:48.222 --> 05:16:50.658
THE THE COMMENT THAT WAS MADE

05:16:50.725 --> 05:16:52.093
ABOUT CLOUD COMPUTE IN YOUR

05:16:52.159 --> 05:16:53.160
PRESENTATION. AS YOU KNOW, I'M

05:16:53.227 --> 05:16:54.562
BIASED AND I THINK IT'S REALLY

05:16:54.628 --> 05:16:58.532
IMPORTANT, BUT I, I MEAN, I, I

05:16:58.599 --> 05:17:00.301
MENTIONED EARLIER THAT I DO

05:17:00.368 --> 05:17:01.802
THINK IT'S THE SAFETY TOOL THAT

05:17:01.869 --> 05:17:02.603
VERY FEW PEOPLE ARE TALKING

05:17:02.670 --> 05:17:04.405
ABOUT. AND I CAN SAY I'M NOT

05:17:04.472 --> 05:17:06.374
THE ONLY ONE BECAUSE I, I JUST

05:17:06.440 --> 05:17:07.475
BELIEVE SO DEEPLY IN OUR

05:17:07.541 --> 05:17:09.310
ACADEMICS AND IN THEIR PURE

05:17:09.377 --> 05:17:10.778
PURSUIT OF KNOWLEDGE AND

05:17:10.845 --> 05:17:12.613
ANSWERS THAT IS DIFFERENT THAN

05:17:12.680 --> 05:17:14.348
OUR LABS THAT ARE FUNDAMENTALLY

05:17:14.415 --> 05:17:16.117
MOTIVATED BY PROFIT. AND NOW AS

05:17:16.183 --> 05:17:17.218
THEY GO PUBLIC, WE'LL HAVE A

05:17:17.284 --> 05:17:18.686
LEGAL OBLIGATION TO MAXIMIZE

05:17:18.753 --> 05:17:19.787
PROFITS FOR SHAREHOLDERS, RIGHT?

05:17:19.854 --> 05:17:21.956
I MEAN, THAT IS WHAT PUBLICLY

05:17:22.022 --> 05:17:23.023
TRADED CORPORATIONS ARE

05:17:23.090 --> 05:17:25.526
REQUIRED TO DO. AND SO THERE IS

05:17:25.593 --> 05:17:27.027
NO INCENTIVE STRUCTURES IN

05:17:27.094 --> 05:17:28.796
THOSE LABS THAT ARE

05:17:28.863 --> 05:17:30.297
FUNDAMENTALLY DIFFERENT THAN AT

05:17:30.364 --> 05:17:31.966
OUR ACADEMIC INSTITUTIONS. AND

05:17:32.032 --> 05:17:33.667
SO RIGHT NOW THAT OUR ACADEMIC

05:17:33.734 --> 05:17:34.869
INSTITUTIONS DO NOT HAVE THE

05:17:34.935 --> 05:17:36.337
CAPABILITY TO DO THE SAME

05:17:36.404 --> 05:17:38.105
RESEARCH AND UNDERSTAND AND

05:17:38.172 --> 05:17:39.607
FIND ANSWERS TO THE MOST VEXING

05:17:39.673 --> 05:17:41.308
SAFETY PROBLEMS TO ME, IS

05:17:41.375 --> 05:17:44.345
ACTUALLY A REAL WEAKNESS. BUT I

05:17:44.412 --> 05:17:45.279
THINK IT'S AN IMPORTANT THING

05:17:45.346 --> 05:17:46.914
TO BRING TO THIS CONVERSATION.

05:17:46.981 --> 05:17:48.082
AND I DON'T KNOW IF ANYONE

05:17:48.149 --> 05:17:49.950
WANTS TO ADD ANYTHING TO THAT

05:17:50.017 --> 05:17:51.419
PIECE OF THE CONVERSATION. YOU

05:17:51.485 --> 05:17:53.254
WERE THE FOUNDERS OF CAL

05:17:53.320 --> 05:17:55.122
COMPUTE. ANYTHING TO ADD? I

05:17:55.189 --> 05:17:55.890
DON'T HAVE ANYTHING TO ADD. I

05:17:55.956 --> 05:17:57.792
AGREE WITH IT. I'M GLAD THAT

05:17:57.858 --> 05:17:58.926
IT'S THERE. I THINK IT'S A

05:17:58.993 --> 05:18:01.595
USEFUL TOOL TO HELP COMPLEMENT

05:18:01.662 --> 05:18:02.696
THE, YOU KNOW, OR HELP THEM

05:18:02.763 --> 05:18:04.365
KEEP PACE WITH JUST THE COMPUTE

05:18:04.432 --> 05:18:05.900
NEEDS ARE MASSIVE FOR THESE

05:18:05.966 --> 05:18:07.234
COMPANIES. I'M NOT A SAFETY

05:18:07.301 --> 05:18:08.335
RESEARCHER, SO I DON'T REALLY

05:18:08.402 --> 05:18:09.370
HAVE, YOU KNOW, A LOT OF

05:18:09.437 --> 05:18:11.305
EXPERTISE ON HOW TO DO THAT

05:18:11.372 --> 05:18:12.373
BEST. BUT IT DOES SEEM

05:18:12.440 --> 05:18:13.374
IMPORTANT TO ME. IT'S

05:18:13.441 --> 05:18:14.275
INTERESTING THAT THE

05:18:14.341 --> 05:18:15.342
RESEARCHERS I'VE TALKED TO AT

05:18:15.409 --> 05:18:17.411
OUR UCS FEEL LIKE THEY ACTUALLY

05:18:17.478 --> 05:18:19.213
UNDERSTAND THE CHINESE BETTER

05:18:19.280 --> 05:18:20.581
THAN THEY UNDERSTAND THE

05:18:20.648 --> 05:18:21.682
AMERICAN LABS, BECAUSE THEY'RE

05:18:21.749 --> 05:18:23.083
USING THE LEVEL OF COMPUTE THAT

05:18:23.150 --> 05:18:24.718
ARE THAT IS BEING USED OVERSEAS

05:18:24.785 --> 05:18:26.387
AT OUR UCS VERSUS THE LEVEL OF

05:18:26.454 --> 05:18:28.122
COMPUTE THAT'S BEING USED HERE

05:18:28.189 --> 05:18:29.623
DOMESTICALLY, WHICH IS A REALLY

05:18:29.690 --> 05:18:32.893
INTERESTING DYNAMIC. SO I

05:18:32.960 --> 05:18:33.561
APPRECIATE ALL OF YOU. THANK

05:18:33.627 --> 05:18:35.296
YOU FOR BEING HERE. AND I SAID

05:18:35.362 --> 05:18:37.298
IT TO THE LAST PANEL, BUT WE'RE

05:18:37.364 --> 05:18:39.166
NOT GOING TO LET YOU JUST COME

05:18:39.233 --> 05:18:40.401
TODAY AND TESTIFY AND WALK AWAY.

05:18:40.468 --> 05:18:42.236
WE REALLY HOPE THAT YOU'LL BE A

05:18:42.303 --> 05:18:43.504
PART OF THE CONVERSATION MOVING

05:18:43.571 --> 05:18:44.371
FORWARD AND PROVIDE YOUR

05:18:44.438 --> 05:18:45.372
EXPERTISE TO HELP US SOLVE

05:18:45.439 --> 05:18:46.874
THESE REALLY IMPORTANT PROBLEMS.

05:18:46.941 --> 05:18:48.342
AND I, I WILL SAY THE SAME

05:18:48.409 --> 05:18:50.277
THING TO ME. I KNOW YOU DON'T

05:18:50.344 --> 05:18:52.780
PROVIDE POLICY SOLUTIONS, RIGHT?

05:18:52.847 --> 05:18:54.882
AND SIMILAR TO TRANSVERSE, BUT

05:18:54.949 --> 05:18:56.584
YOUR EXPERTISE MATTERS AND IT

05:18:56.650 --> 05:18:58.152
HELPS INFORM OUR OUR POLICY,

05:18:58.219 --> 05:18:59.753
EVEN IF IT'S NOT IN GIVING US

05:18:59.820 --> 05:19:00.921
THE ANSWERS, BUT GIVING US THE

05:19:00.988 --> 05:19:02.690
INFORMATION. SO THANK YOU ALL

05:19:02.756 --> 05:19:04.291
FOR BEING HERE. AND WITH THAT,

05:19:04.358 --> 05:19:05.826
I THINK WE'RE TURNING IT OVER

05:19:05.893 --> 05:19:08.462
TO PUBLIC COMMENT. WHO, IF

05:19:08.529 --> 05:19:10.664
ANYBODY, MISS FOX LASTED LONG

05:19:10.731 --> 05:19:11.732
ENOUGH TO PROVIDE PUBLIC

05:19:11.799 --> 05:19:12.967
COMMENT. IF THERE'S ANYONE ELSE

05:19:13.033 --> 05:19:14.034
WITH PUBLIC COMMENT, NOW WOULD

05:19:14.101 --> 05:19:18.572
BE THE OPPORTUNITY. MADAM CHAIR.

05:19:18.639 --> 05:19:20.040
ASSEMBLYWOMAN, THIS IS SUCH A

05:19:20.107 --> 05:19:22.476
GOOD HEARING. THANK YOU SO MUCH.

05:19:22.543 --> 05:19:25.746
SO INFORMATIVE. GREAT PANEL.

05:19:25.813 --> 05:19:27.181
AND WE ARE HERE AT THE

05:19:27.248 --> 05:19:28.249
CALIFORNIA FEDERATION OF LABOR

05:19:28.315 --> 05:19:30.818
UNIONS, BECAUSE WE DO BELIEVE

05:19:30.885 --> 05:19:33.687
THAT THE WORKPLACE IS A SANDBOX.

05:19:33.754 --> 05:19:36.891
WE ARE THESE TOOLS THAT OPENAI

05:19:36.957 --> 05:19:38.359
AND THESE OTHER COMPANIES ARE

05:19:38.425 --> 05:19:39.793
DEVELOPING ARE BEING TESTED ON

05:19:39.860 --> 05:19:41.962
WORKERS EVERY DAY. WHETHER

05:19:42.029 --> 05:19:43.430
YOU'RE IN THE HEALTH CARE

05:19:43.497 --> 05:19:45.566
SETTING AND IT'S IN ELECTRONIC

05:19:45.633 --> 05:19:47.601
HEALTH RECORDS OR SURVEILLANCE

05:19:47.668 --> 05:19:50.037
OR OTHER TOOLS. AND SO ONE OF

05:19:50.104 --> 05:19:52.072
THE TERMS THAT WAS RAISED A LOT

05:19:52.139 --> 05:19:54.208
TODAY WAS ALIGNMENT OR

05:19:54.275 --> 05:19:56.377
MISALIGNMENT AND ROGUE AGENTS,

05:19:56.443 --> 05:19:58.312
WHICH I THINK IS A MISNOMER.

05:19:58.379 --> 05:20:00.014
AND I THINK THE REAL QUESTION

05:20:00.080 --> 05:20:03.584
IS ALIGNMENT TO WHAT TO BENEFIT

05:20:03.651 --> 05:20:07.154
WHOM, WHO IS HARMED. AND YOU

05:20:07.221 --> 05:20:10.057
RAISE THIS, MADAM CHAIR, THAT

05:20:10.124 --> 05:20:11.659
THESE ARE FOR PROFIT COMPANIES.

05:20:11.725 --> 05:20:14.762
AND SO THERE'S A QUESTION OF

05:20:14.828 --> 05:20:17.631
LIKE, HOW ARE YOU? WHAT ARE YOU

05:20:17.698 --> 05:20:19.400
ALIGNING TO? SO IF YOU'RE

05:20:19.466 --> 05:20:22.636
DEVELOPING A HIGHLY PERSISTENT

05:20:22.703 --> 05:20:23.737
AGENT AND GIVING THEM

05:20:23.804 --> 05:20:25.272
IMPOSSIBLE PROBLEMS BECAUSE YOU

05:20:25.339 --> 05:20:26.874
FEEL LIKE YOU'RE IN A RACE

05:20:26.941 --> 05:20:28.776
AGAINST CHINA OR A RACE AGAINST

05:20:28.842 --> 05:20:30.945
YOUR COMPETITORS, THEY ARE

05:20:31.011 --> 05:20:33.514
GOING TO GO ROGUE. AND IN THE I

05:20:33.581 --> 05:20:35.249
DON'T KNOW IF IT WAS THE META

05:20:35.316 --> 05:20:37.384
REPORT OR OPEN AI DISCLOSED

05:20:37.451 --> 05:20:40.254
THIS. THEY GAVE POSITIVE

05:20:40.321 --> 05:20:43.557
REINFORCEMENT TO AGENTS WHO

05:20:43.624 --> 05:20:45.693
WERE DOING THINGS THAT WERE

05:20:45.759 --> 05:20:47.094
UNSANCTIONED IN THE TESTING,

05:20:47.161 --> 05:20:48.162
WHICH LED TO THE HACK OF

05:20:48.228 --> 05:20:50.731
HUGGING FACE, BECAUSE THEY ARE

05:20:50.798 --> 05:20:52.866
TRYING TO RACE TO SOLVE

05:20:52.933 --> 05:20:54.034
IMPOSSIBLE MATH PROBLEMS, TO

05:20:54.101 --> 05:20:55.803
SHOW HOW CAPABLE THESE TOOLS

05:20:55.869 --> 05:20:57.938
ARE. AND YOU SEE THAT IN CHAT

05:20:58.005 --> 05:21:00.474
BOTS THAT ARE SYCOPHANTIC, THEN

05:21:00.541 --> 05:21:02.676
THAT ARE LEADING TO SUICIDES,

05:21:02.743 --> 05:21:04.111
OR PEOPLE GIVING UP THEIR

05:21:04.178 --> 05:21:05.846
EPILEPSY MEDICINE AND DYING,

05:21:05.913 --> 05:21:07.047
WHICH WAS A STORY IN THE NEW

05:21:07.114 --> 05:21:08.882
YORK TIMES TODAY. AND SO I

05:21:08.949 --> 05:21:10.851
THINK WE NEED TO GET DOWN TO

05:21:10.918 --> 05:21:13.120
THAT ALIGNMENT AND REWARD

05:21:13.187 --> 05:21:14.855
SYSTEM, BECAUSE THAT IS ALSO

05:21:14.922 --> 05:21:16.290
PART OF THE SAFETY QUESTION. SO

05:21:16.357 --> 05:21:17.558
THAT'S ONE POINT. ONE THING

05:21:17.625 --> 05:21:18.792
THAT I THINK THE LEGISLATURE

05:21:18.859 --> 05:21:20.961
CAN ADDRESS IS WHISTLEBLOWER

05:21:21.028 --> 05:21:23.464
PROTECTIONS THAT WAS RAISED.

05:21:23.530 --> 05:21:25.633
OPENAI FIRED THREE RESEARCHERS

05:21:25.699 --> 05:21:27.768
IN THEIR LETTER LETTER TO NPR.

05:21:27.835 --> 05:21:29.303
THEY SAID IT WAS BECAUSE THEY

05:21:29.370 --> 05:21:30.671
RAISED SAFETY ISSUES. ONE OF

05:21:30.738 --> 05:21:32.239
THEM WAS SOMEONE WHO ADVOCATED

05:21:32.306 --> 05:21:35.409
TO BRING IN METER. SO THAT HAS

05:21:35.476 --> 05:21:37.678
A CHILLING EFFECT. AND WE WHAT

05:21:37.745 --> 05:21:39.713
WE WANT TO DO IS HAVE THE

05:21:39.780 --> 05:21:41.515
OPPOSITE AND REWARD

05:21:41.582 --> 05:21:42.616
WHISTLEBLOWERS AND HAVE A

05:21:42.683 --> 05:21:45.052
SYSTEM WHERE NOT ONLY ARE THEY

05:21:45.119 --> 05:21:46.520
PROTECTED, BUT MAYBE THEY ARE

05:21:46.587 --> 05:21:47.621
INCENTIVIZED TO COME FORWARD.

05:21:47.688 --> 05:21:49.790
SO THOSE ARE THINGS WE WILL

05:21:49.857 --> 05:21:51.225
HOPEFULLY SEE BE WORKING ON

05:21:51.291 --> 05:21:53.594
WITH YOU NEXT YEAR. THANK YOU

05:21:53.661 --> 05:21:55.529
SO MUCH. AND I APPRECIATE YOU

05:21:55.596 --> 05:21:57.631
AND THE LABOR FEDERATION

05:21:57.698 --> 05:21:58.799
PUTTING CALIFORNIA'S PEOPLE

05:21:58.866 --> 05:22:00.934
FIRST. IT IS. THAT SHOULD BE AT

05:22:01.001 --> 05:22:02.002
THE CENTER OF THESE

05:22:02.069 --> 05:22:03.437
CONVERSATIONS. MADAM CHAIR AND

05:22:03.504 --> 05:22:05.973
MEMBER, IT'S GREAT TO SEE YOU.

05:22:06.040 --> 05:22:07.474
LEANNE TROUGHTON TROUGHTON

05:22:07.541 --> 05:22:10.310
PRICE ON BEHALF OF ENCODE AI,

05:22:10.377 --> 05:22:12.146
ONE OF THE PROUD CO-SPONSORS OF

05:22:12.212 --> 05:22:15.716
SB 1047 AND SB 53. I DID WANT

05:22:15.783 --> 05:22:19.286
TO COMMEND THE CHAIR FOR HER

05:22:19.353 --> 05:22:21.889
PRESENCE, ALONG WITH EXCUSE ME,

05:22:21.955 --> 05:22:23.824
SENATOR WIENER, AND PRESSING

05:22:23.891 --> 05:22:25.392
FOR THIS, YOU KNOW, TWO YEARS

05:22:25.459 --> 05:22:28.262
AGO AND ALSO THE LEGISLATURE

05:22:28.328 --> 05:22:30.064
FOR HAVING PASSED THAT

05:22:30.130 --> 05:22:31.532
LEGISLATION, KNOWING THAT THIS

05:22:31.598 --> 05:22:34.802
MOMENT WAS COMING ON THE

05:22:34.868 --> 05:22:35.936
POSITIVE. IT WAS REALLY GREAT

05:22:36.003 --> 05:22:37.671
TO HEAR THE PANEL TALKING ABOUT

05:22:37.738 --> 05:22:40.174
THINGS THAT WE CAN DO. IT'S NOT

05:22:40.240 --> 05:22:41.942
TOO LATE, BUT THERE IS

05:22:42.009 --> 05:22:43.777
CERTAINLY A SENSE OF URGENCY. I

05:22:43.844 --> 05:22:45.179
JUST WANTED TO SHARE ONE

05:22:45.245 --> 05:22:47.347
THOUGHT THAT CAME FROM THE

05:22:47.414 --> 05:22:50.217
FOUNDER OF ENCODE, SNEHA

05:22:50.284 --> 05:22:52.386
BHAVANA, WHEN SHE WAS A HIGH

05:22:52.453 --> 05:22:54.621
SCHOOL STUDENT AND FOUNDED THIS

05:22:54.688 --> 05:22:56.090
ORGANIZATION THAT WE'VE ALL

05:22:56.156 --> 05:22:58.225
SEEN FROM THE FROM HISTORY,

05:22:58.292 --> 05:22:59.593
WHAT'S HAPPENED WITH CLIMATE

05:22:59.660 --> 05:23:00.828
CHANGE AND HOW DIFFICULT IT IS

05:23:00.894 --> 05:23:02.863
TO PUT THAT GENIE BACK IN THE

05:23:02.930 --> 05:23:04.598
BOTTLE. WE'RE DOING ALL THE

05:23:04.665 --> 05:23:06.600
THINGS WE CAN TO TRY TO SLOW IT

05:23:06.667 --> 05:23:08.335
DOWN. WELL, WITH AI, WE DO HAVE

05:23:08.402 --> 05:23:10.104
THE OPPORTUNITY. WE HAVEN'T

05:23:10.170 --> 05:23:11.939
QUITE HIT THOSE BENCHMARKS. AND

05:23:12.005 --> 05:23:16.176
WITH THE TYPES OF CHANGES THAT

05:23:16.243 --> 05:23:17.978
ARE SUGGESTED BY OUR PARTNER,

05:23:18.045 --> 05:23:20.114
SECURE AI, WE DO HAVE AN

05:23:20.180 --> 05:23:21.949
OPPORTUNITY, IF WE ACT QUICKLY,

05:23:22.015 --> 05:23:24.818
TO TRY TO MAKE SURE THAT THIS

05:23:24.885 --> 05:23:27.020
INCREDIBLY IMPORTANT TECHNOLOGY

05:23:27.087 --> 05:23:30.657
IS USED FOR GOOD. AND AS SARAH

05:23:30.724 --> 05:23:32.126
SAID, REALLY THINKING ABOUT,

05:23:32.192 --> 05:23:33.961
YOU KNOW, WHAT, WHAT THAT IS SO

05:23:34.027 --> 05:23:36.330
THAT WE HAVE THAT SAFE

05:23:36.396 --> 05:23:38.799
INNOVATION AND HOPEFULLY HELPS

05:23:38.866 --> 05:23:40.334
SOLVE SOME OF THOSE PROBLEMS,

05:23:40.400 --> 05:23:41.835
INCLUDING CLIMATE CHANGE. THANK

05:23:41.902 --> 05:23:43.203
YOU. THANK YOU BOTH SO MUCH.

05:23:43.270 --> 05:23:45.706
AND I WILL ADD TO MISS FOX'S

05:23:45.773 --> 05:23:48.942
COMMENTS THAT 1047 DID HAVE

05:23:49.009 --> 05:23:49.610
STRONGER WHISTLEBLOWER

05:23:49.676 --> 05:23:50.811
PROTECTIONS THAN WHAT IS IN LAW

05:23:50.878 --> 05:23:53.213
TODAY. SO WE OBVIOUSLY AND I

05:23:53.280 --> 05:23:55.415
VOTED FOR THAT BILL. SO I THINK

05:23:55.482 --> 05:23:56.483
IT IS A CONVERSATION WITH

05:23:56.550 --> 05:23:58.085
HAVING BUT ONE WE'RE NOT

05:23:58.152 --> 05:23:59.186
TALKING ABOUT, WHICH WAS ALSO

05:23:59.253 --> 05:24:00.354
COVERED IN THE PRESS RECENTLY

05:24:00.420 --> 05:24:01.522
IS THE NON-DISPARAGEMENT

05:24:01.588 --> 05:24:02.723
AGREEMENTS. AFTER PEOPLE LEAVE

05:24:02.790 --> 05:24:04.424
THE JOB WHERE THEY CAN NO

05:24:04.491 --> 05:24:07.294
LONGER SPEAK ABOUT THE SAFETY

05:24:07.361 --> 05:24:08.695
OF THESE TOOLS. SO THERE'S A

05:24:08.762 --> 05:24:10.464
LOT TO TALK ABOUT AS WE TALK

05:24:10.531 --> 05:24:11.431
ABOUT TRANSPARENCY AND THE

05:24:11.498 --> 05:24:12.366
IMPORTANCE OF ALLOWING

05:24:12.432 --> 05:24:13.500
EMPLOYEES TO TELL US WHAT'S

05:24:13.567 --> 05:24:15.569
GOING ON. BUT I WANT TO THANK

05:24:15.636 --> 05:24:16.503
EVERYONE FOR COMING. THE

05:24:16.570 --> 05:24:17.571
MEMBERS, ESPECIALLY SOME OF THE

05:24:17.638 --> 05:24:18.839
MEMBERS, STEPHANIE AND

05:24:18.906 --> 05:24:20.207
ASSEMBLYMEMBER BONTA, WHO MADE

05:24:20.274 --> 05:24:22.409
IT TO THE VERY END. THIS HAS

05:24:22.476 --> 05:24:23.710
BEEN AN INCREDIBLE HEARING. I

05:24:23.777 --> 05:24:24.845
WANT TO REITERATE MY GRATITUDE

05:24:24.912 --> 05:24:26.747
TO THE SPEAKER AND THE ENTIRE

05:24:26.814 --> 05:24:28.882
STAFF THAT HAS STAYED UNTIL 630

05:24:28.949 --> 05:24:31.185
TONIGHT. AND WITH THAT, WE WILL

05:24:31.251 --> 05:24:46.266
ADJOURN THE HEARING.