[ fSrSSKrSSKJr SSKJr SSKJr SSKJ r SSK J r SSK Jr SS KJr SS KJr SS KJr SSKJr SS KJr SS KJr SSKJr SSKJr SSKJr SrSSSSSSSSSSS. rSSSS S!S"S#S$S%S&S. rS'rS(rS)r S*r!S+r"S7S,jr#S7S-jr$S.r%S/r&S0r'S1r(S2r)S3r*S4r+S5r,S6r-g)8z-Utilities for the model garden command group.N)encoding) exceptions) operations)client)metric) quota_info)endpoints_util) models_util)operations_util)log) properties)requests) resources) console_io?+CustomModelServingP4GPUsPerProjectPerRegion+CustomModelServingT4GPUsPerProjectPerRegion+CustomModelServingL4GPUsPerProjectPerRegion,CustomModelServingK80GPUsPerProjectPerRegion-CustomModelServingV100GPUsPerProjectPerRegion-CustomModelServingP100GPUsPerProjectPerRegion-CustomModelServingA100GPUsPerProjectPerRegion1CustomModelServingA10080GBGPUsPerProjectPerRegion-CustomModelServingH100GPUsPerProjectPerRegion+CustomModelServingV5ETPUPerProjectPerRegion) NVIDIA_TESLA_P4NVIDIA_TESLA_T4 NVIDIA_L4NVIDIA_TESLA_K80NVIDIA_TESLA_V100NVIDIA_TESLA_P100NVIDIA_TESLA_A100NVIDIA_A100_80GBNVIDIA_H100_80GBTPU_V5_LITEPOD#custom_model_serving_nvidia_p4_gpus#custom_model_serving_nvidia_t4_gpus#custom_model_serving_nvidia_l4_gpus$custom_model_serving_nvidia_k80_gpus%custom_model_serving_nvidia_v100_gpus%custom_model_serving_nvidia_p100_gpus%custom_model_serving_nvidia_a100_gpus*custom_model_serving_nvidia_a100_80gb_gpus%custom_model_serving_nvidia_h100_gpuscustom_model_serving_tpu_v5eametric.type="serviceruntime.googleapis.com/quota/allocation/usage" AND resource.type="consumer_quota" AND metric.label.quota_metric="aiplatform.googleapis.com/{}" AND resource.label.project_id="{}" AND resource.label.location="{}" AND resource.label.service="aiplatform.googleapis.com"c[RRUU[RR R RS.SS9$)zNsK    ! !$"))..66@@; " c[R"USSS[U5nURH7nURSU:XdMUR R =(d Ss $ g)z >sC H  r?cU(aSUSU3RSS5S[$SUSUSU3RSS5S[$)Nrzr{r|r}z publishers-z-models-r~rs r=GetOneClickEndpointLabelValuersq  * . 6 6sC @   n%Xj\;M:NO    % $  r?c[R"5RSUSUS35nURS:wa[R "S5e[ UR5S5$)z9Checks if the HF model is gated or not by calling HF API."https://huggingface.co/api/models//z ?blobs=truezfSomething went wrong when we call HuggingFace's API to get the model metadata. Please try again later.gated)r GetSessionget status_codecore_exceptions InternalErrorbooljson)rr hf_responses r=IsHFModelGatedrsq##%))*>*:!J<{S+#  ' ' 3  k ) **r?c[R"5RSSUSUS3SSU30S9nURS:wa[R "S 5eg) NGETrrz /auth-check AuthorizationzBearer )headersrzaThe Hugging Face access token is not valid or does not have permission to access the gated model.)rrrequestrrError)hf_tokenrrrs r=VerifyHFTokenPermissionrsr##%-- *>*:!J<{S'( 45.+ #    &  r?cURRRnSnUR (d#UR (dUR(GaUHnUR (a.URRRUR :wdUR (aE[URRR5UR R5:wd5UR(a&URRUR:waMUn O U(d[R"S5e[ R"R%S5 O$USn[ R"R%S5 URRnURRnUR(a,[ R"R%SUR35 UR(a,[ R"R%SUR35 UR&(a,[ R"R%S UR&35 U(a"[ R"R%S U35 U$![a [R"S5ef=f) zGReturns a best suited deployment configuration for the publisher model.zModel does not support deployment, please use a deploy-able model instead. You can use the `gcloud ai model-garden models list` command to find out which ones are currently supported by the `deploy` command.NzThe machine type, accelerator type and/or container image URI is not supported by the model. You can use `gcloud alpha/beta ai model-garden models list-deployment-config` command to find the supported configurationsz,Using the selected deployment configuration:rz+Using the default deployment configuration:z Machine type: z Accelerator type: z Accelerator count: z Container image URI: )supportedActionsmultiDeployVertexAttributeErrorrr machine_typerHcontainer_image_uridedicatedResources machineSpec machineTypestracceleratorTypeupper containerSpecimageUrir statusPrintacceleratorCount)argspublisher_model multi_deploy deploy_configdeploy machine_specrs r=GetDeployConfigrs) ((::LL- $//43K3K3K++77CC""###f//;;KKL&&,,./&&&&//43K3KK m '*   ! ! &  JJCD!OMJJBC11==,%33<<JJ|'?'?&@AB!!JJ*<+G+G*HIJ""JJ+L,I,I+JKLJJ-.A-BCD m      s J!J?c US:XaSnSnO US:XaSnSn[5(d&[U5S:Xa[R"S5 g[R R RR5n[URXB5nXS:a4[R"S [US URS US US 3 5e[URXB5nXc-U:a7[R"S [US URSUSUS US 3 5e[RR!SUSURSUSUS3 5 g)z8Checks the accelerator quota for the project and region.zct5lp-hightpu-1tr%zct5lp-hightpu-4tNonez@No accelerator type specified, skipping accelerator quota check.Nz+The project does not have enough quota for z in z) to deploy the model. The quota limit is z and you are requesting for z. Please use a different region or request more quota by following https://cloud.google.com/vertex-ai/docs/quotas#requesting_additional_quota.z+ to deploy the model. The current usage is z out of zNThe project has enough quota. The current usage of quota for accelerator type z in region z is r|)rxrr debugr r7r8r9r:rKrGrrrgrrrr)rrrHaccelerator_countr9 quota_limitrqs r=CheckAcceleratorQuotars '''))'    & ii L    " " * * 4 4 6't{{GF+$    12B C DE KK=11< >,-.W  W !gH-&4    12B C DE KK=33@/B =,-.W  W  **+,K }E  Q0r?c RURUUURRRURRRR SUS9/S9S9n[ R "UU[R"UR55nUc[R"S5e[R"U5nSU;a[R"S5e[RR!SR#US55 USR%S5S $) z)Creates a Vertex endpoint for deployment.z mg-cli-deploy)keyrF)additionalProperties)labelszEInternal error: Failed to create a Vertex endpoint. Please try again.namezOCreated Vertex AI endpoint: {}. Starting to upload the model to Model Registry.r) CreateBetamessages$GoogleCloudAiplatformV1beta1Endpoint LabelsValueAdditionalPropertyr WaitForOpMayber ParseOperationrrrrMessageToPyValuer rrrfsplit) endpoint_name label_value region_refoperation_clientendpoints_clientcreate_endpoint_opcreate_endpoint_response_msgresponses r=CreateEndpointrFs2(22  & & K K W W''LLXXkk%[l X3 "1!?!?##$6$;$;<" ")  ' 'O  & &'C D( 8  ' 'O ** x  &    $R ((r?cSupxn URR(aVURRV s0sHoRU R_M nn U(aSU;aURUS'URR (a[ URR 5nURR(a[ URR5n [R"5n U RURRR5USSURURRU UUURR SR"/SURR$URR&U R(R+U R(R-SR/U(aSU-OUU5S9S9S 9n [0R2"[4R6"5U [8R:"U R5S 9n U c[<R>"S 5e[@RB"U 5nS U;a[<R>"S 5e[DRFRIS R/US 55 US RKS5S$s sn f)z1Uploads the Model Garden model to Model Registry.)NNNHUGGING_FACE_HUB_TOKENNrzpublishers/{}/models/{}rz)publicModelName)modelGardenSource)base_model_source)operations_clientopop_refz`Internal error: Failed to upload a Model Garden model to Model Registry. Please try again later.modelzEUploaded model to Model Registry at {}. Starting to deploy the model.rr)&renvrrFhugging_face_access_tokenrlistcommand client_models ModelsClient UploadV1Beta1CONCEPTSrGr6 artifactUrirports containerPort predictRoute healthRouter0GoogleCloudAiplatformV1beta1ModelBaseModelSource-GoogleCloudAiplatformV1beta1ModelGardenSourcerfr rrOperationsClientr ParseModelOperationrrrrr rrr)rrrequires_hf_tokenruploaded_model_namerpublisher_model_namecontainer_env_varscontainer_argscontainer_commandsvar models_clientupload_model_opupload_model_response_msgupload_model_responses r= UploadModelrps>,7%.(^&a `0/2.<<"335   , ,_-A-A B &  ' ' - #334MN ))  ' ' - ** $W-.  w ' - -c 22 66ys K c URRRnURRRnSn UcUb80n Ub,[ U5R 5R SS5U S'UbXS'UR[X R5UURUURRRU SSS9n [R"UU [R"U R5SS9 U RR!S 5S n [#S US U S URSUS URS3 5 g)z8Deploys the Model Registry model to the Vertex endpoint.Nr}r{typecountT)raccelerator_dictenable_access_loggingenable_container_logging) asynchronousrrDeploying the model to the endpoint. To check the deployment status, you can try one of the following methods: 1) Look for endpoint `g` at the [Vertex AI] -> [Online prediction] tab in Cloud Console 2) Use `gcloud ai operations describe --region=zc` to find the status of the deployment long-running operation 3) Use `gcloud ai endpoints describe z+` command to check the endpoint's metadata.)rrrrrlowerr^DeployModelBetar>rGrr rr rrrprint) rrr;rmodel_idrrrHrrdeploy_model_op deploy_op_ids r= DeployModelr sj&&22BB&&22CC!%6%B#!$%5!6!!F!FsC!Pv$"3w$44[++. kk 33??KK' #5 /  ##O$8$89  !%%++C04,!?#. 4;;-8(MDKK=A+ + r?c rUb%URnURnURnOSnSnSnUR"S90S[R R RR5_SUR_SU_SUR_SU_SU_SU_S U_S UR_S UR_S UR_S UR_SUR_SUR _SUR"_SUR$_SUR&_SUR(_SUR*_SUR,_SUR._SUR0_SUR2_SUR4_SUR6_SUR8_SUR:_SUR<_SUR>_SUR@_6n U R^RWS(5S2n [NRPRSS3US4U S5URS635 [`Rb"UU [eU R^5URfS7S89 g![BRDGa`n U RFS :XaqS!U RH;aaURcTUR(d.[JRL"S"5Ul UR(dM.[UUUUUU5 Sn A gU RFS#:XaS$U RH;a[NRPRSS%5 URTRWS&5S'RWS(5up[JRX"S)U S*U S+3S,S-S.S/S.S09Ul O.![JRZa [\RZ"S15ef=f[UUUUUU5 Sn A gU eSn A ff=f):z1Deploys the publisher model to a Vertex endpoint.Nr9locationr accept_eularHrrendpoint_display_namerspotreservation_affinityuse_dedicated_endpointdisable_dedicated_endpointenable_fast_tryoutrcontainer_commandrrcontainer_portscontainer_grpc_portscontainer_predict_routecontainer_health_route$container_deployment_timeout_secondscontainer_shared_memory_size_mbcontainer_startup_probe_exec&container_startup_probe_period_seconds'container_startup_probe_timeout_secondscontainer_health_probe_exec%container_health_probe_period_seconds&container_health_probe_timeout_secondsiz)provide a valid Hugging Face access tokenz2Please enter your Hugging Face read access token: iEULAzIThe End User License Agreement (EULA) of the model has not been accepted.@rrzThe model can be deployed only if the EULA of the model has been accepted. You can view it at https://console.cloud.google.com/vertex-ai/publishers/z/model-garden/z):zDo you want to accept the EULA?FTzEULA is not accepted.)message prompt_stringrv cancel_on_no cancel_stringthrow_if_unattendedz6Please accept the EULA using the `--accept-eula` flag.rrrrz>` to find the status of the deployment long-running operation i6)r max_wait_msrw)4rrrDeployr r7r8r9r:rGrrrrrrrrrrrrrrrrrrrrr r!r"apitools_exceptions HttpErrorrcontentrPromptPasswordr rrrrPromptContinuerrrr rrr)rrrrr mg_clientrrHr deploy_ope publisherr r s r=r+r+s'_!--l%55&77l  !!&&..88:$$  *  , ",#'"@"@YY"66 $::$(#B#B 22!44 00!"**#$ 22%&,,'("66)*!% < <+, $::-..2-V-V/0)-(L(L12&*%F%F3404/Z/Z56150\0\78%)$D$D9:/3.X.X;<04/Z/Z=Ip%%c*2.,**!?#. 4;;-84 4  Y^^$$$ C  & &6  7199 D  * * 2..)3)B)B @* &...        # &AII"5 jj 8!JJ,,S1!4::3?i %44JKTTbckbllnp <1 $     ## D           gm6sDF8IN6A*N1N1A.N1%M,+N1,+NN1/N11N6c>[RRUSS9$)zParse operation resource to the operation reference object. Args: operation_name: The operation resource to wait on Returns: The operation reference object z(aiplatform.projects.locations.operations)r4)rr5ParseRelativeName)operation_names r=rrhs(    - -; . r?)r8).__doc__rXapitools.base.pyrrr,googlecloudsdk.api_lib.air googlecloudsdk.api_lib.ai.modelsrr!googlecloudsdk.api_lib.monitoringrgooglecloudsdk.api_lib.quotasrgooglecloudsdk.command_lib.air r r googlecloudsdk.corerr r rrgooglecloudsdk.core.consolerrrBrgrer>rKrrrxrrrrrrrrr r+rrwr?r=rBs4%>0D44859=#*()2ED>FHHHKGC %!=<6>@@@D?4 )%:   #LDF DF$ +  <~3l')TH7V3ltn r?