o g@; @sUdZddlZddlZddlZddlZddlmZmZddlm Z ddl m Z ddl m Z ddlmZmZmZmZmZmZmZmZmZmZmZmZmZmZddlmZdd lm Z m!Z!m"Z"m#Z#m$Z$m%Z%d d l&m'Z'm(Z(m)Z)m*Z*d d l+m,Z,m-Z-erddl.m/Z/m0Z0ddl1m2Z2e3Z4ee3e fZ5ee6efZ7ee7e5e4fZ8ddhZ9e:e;Ze GdddZ?ddZ@ddZAddZBede8dee7fd d!ZCededdeedfd"d!ZCe dee8deee7ddffd#d!ZCde8de3fd$d%ZDd&e3dd'fd(d)ZEde6defd*d+ZFde6defd,d-ZGde6dd'fd.d/ZHd0ee6efdefd1d2ZId3ee6d4eJdeee3ee-ffd5d6ZKd3ee6d4eJdeee3ee-ffd7d8ZLd9e6d4eJdeee3e-ffd:d;ZMdZNdr5)r?rAr5r5r6 get_responsefszTaskProviderHelper.get_response)r/r0r1__doc__rrr r2rr(r@rbytesrBr5r5r5r6r7Xs*    r7c@s2eZdZUdZeed<eed<eed<eed<dS) ModelStatusa This Dataclass represents the model status in the Hugging Face Inference API. Args: loaded (`bool`): If the model is currently loaded into Hugging Face's InferenceAPI. Models are loaded on-demand, leading to the user's first request taking longer. If a model is loaded, you can be assured that it is in a healthy state. state (`str`): The current state of the model. This can be 'Loaded', 'Loadable', 'TooBig'. If a model's state is 'Loadable', it's not too big and has a supported backend. Loadable models are automatically loaded when the user first requests inference on the endpoint. This means it is transparent for the user to load a model, except that the first call takes longer to complete. compute_type (`Dict`): Information about the compute resource the model is using or will use, such as 'gpu' type and number of replicas. framework (`str`): The name of the framework that the model was built with, such as 'transformers' or 'text-generation-inference'. loadedstate compute_type frameworkN)r/r0r1rCboolr3r2r r5r5r5r6rEks  rEcCststdddl}|S)NzMPlease install aiohttp to use `AsyncInferenceClient` (`pip install aiohttp`).r)r ImportErroraiohttp)rLr5r5r6_import_aiohttprMcCststdddl}|S)z.Make sure `numpy` is installed on the machine.zGPlease install numpy to use deal with embeddings (`pip install numpy`).rN)rrKnumpy)rOr5r5r6 _import_numpyrNrPcCststdddlm}|S)z,Make sure `PIL` is installed on the machine.zPlease install Pillow to use deal with images (`pip install Pillow`). If you don't want the image to be post-processed, use `client.post(...)` and get the raw response from the server.rr&)r rKPILr'r&r5r5r6_import_pil_images  rRcontentr<cCr=r>r5rSr5r5r6_open_as_binaryrUcCr=r>r5rTr5r5r6rUrVccst|tr2|ds|dr"td|t|jVdSt|}| s2t d|dt|trZtd|| d }|VWddS1sSwYdS|VdS) zOpen `content` as a binary file, either from a URL, a local path, or raw bytes. Do nothing if `content` is None, TODO: handle a PIL.Image as input TODO: handle base64 as input zhttps://zhttp://zDownloading content from NzFile not found at z. If `data` is a string, it must either be a URL or a path to a local file. To pass raw content, please encode it as bytes first.zOpening content from rb) isinstancer2 startswithloggerdebugrgetrSrexistsFileNotFoundErroropen)rSfr5r5r6rUs"    " cCsPt|}t|tr |n|}t|WdS1s!wYdS)z\Encode a raw file (image, audio) into base64. Can be bytes, an opened file, a path or a URL.N)rUrXrDreadbase64 b64encodedecode)rSr- data_as_bytesr5r5r6 _b64_encodes  $rf encoded_imager'cCst}|tt|S)z/Parse a base64-encoded string into a PIL Image.)rRr_ioBytesIOrb b64decode)rgr'r5r5r6 _b64_to_imagesrkcCt|S)acParse bytes from a Response object into a Python list. Expects the response body to be JSON-encoded data. NOTE: This is exactly the same implementation as `_bytes_to_dict` and will not complain if the returned data is a dictionary. The only advantage of having both is to help the user (and mypy) understand what kind of data to expect. r,loadsrdrTr5r5r6_bytes_to_listrocCrl)acParse bytes from a Response object into a Python dictionary. Expects the response body to be JSON-encoded data. NOTE: This is exactly the same implementation as `_bytes_to_list` and will not complain if the returned data is a list. The only advantage of having both is to help the user (and mypy) understand what kind of data to expect. rmrTr5r5r6_bytes_to_dictrprqcCst}|t|S)zParse bytes from a Response object into a PIL Image. Expects the response body to be raw bytes. To deal with b64 encoded images, use `_b64_to_image` instead. )rRr_rhri)rSr'r5r5r6_bytes_to_imagesrrrAcCst|tr t|S|Sr>)rXrDr,rn)rAr5r5r6_as_dictsrsbytes_output_as_linesdetailsc csD|D]}zt||}Wn tyYdSw|dur|VqdS)z*Used in `InferenceClient.text_generation`.N%_format_text_generation_stream_output StopIterationrtru byte_payloadoutputr5r5r6 _stream_text_generation_responses r|c CsN|2z 3dHW}zt||}Wn tyYdSw|dur#|Vq6dS)z/Used in `AsyncInferenceClient.text_generation`.Nrvryr5r5r6&_async_stream_text_generation_responses r}rzcCs~|dsdS|dkrtd|d}t|dd}|ddur2t |d|dt |}|s=|j j S|S Nsdata:s data: [DONE]z[DONE] signal received.zutf-8zdata:z/nerror error_type)rYstriprxrdr,rnlstriprstripr\_parse_text_generation_errorr#parse_obj_as_instancetokentext)rzrupayload json_payloadr{r5r5r6rw+s    rw bytes_linesc csB|D]}zt|}Wn tyYdSw|dur|VqdS)zFUsed in `InferenceClient.chat_completion` if model is served with TGI.N%_format_chat_completion_stream_outputrxritemr{r5r5r6 _stream_chat_completion_responseAs  rc CsL|2z3dHW}zt|}Wn tyYdSw|dur"|Vq6dS)z/Used in `AsyncInferenceClient.chat_completion`.Nrrr5r5r6&_async_stream_chat_completion_responseNs  rcCsn|dsdS|dkrtd|d}t|dd}|ddur2t |d|dt |Sr~) rYrrxrdr,rnrrr\rr"r)rzrrr5r5r6r[s    rclientr%r$cCs4|j2z 3dHW}|Vq6|IdHdSr>)rSrclose)rrArzr5r5r6_async_yield_fromps  r#_UNSUPPORTED_TEXT_GENERATION_KWARGSr+unsupported_kwargscCst|g|dSr>)r setdefaultextend)r+rr5r5r6'_set_unsupported_text_generation_kwargssrcCs t|gSr>)rr\)r+r5r5r6'_get_unsupported_text_generation_kwargss r http_errorcCs^zt|ddp |j}|d}|d}Wn ty |w|dur-t||}|||)z Try to parse text-generation-inference error message and raise HTTPError in any case. Args: error (`HTTPError`): The HTTPError that have been raised. response_error_payloadNrr)getattrrAr,r\ Exceptionr)rrrr exceptionr5r5r6raise_text_generation_errors    rrrcCsH|dkrt|S|dkrt|S|dkrt|S|dkr t|St|S)N generationincomplete_generation overloaded validation)rrrrr)rrr5r5r6rsr)XrCrbrhr,loggingabcrr contextlibr dataclassesrpathlibrtypingrrr r r r r rrrrrrrrequestsrhuggingface_hub.errorsrrrrrrutilsrrrr _generated.typesr"r#rLr$r% PIL.Imager'r2UrlTPathTrDBinaryTr4TASKS_EXPECTING_IMAGES getLoggerr/rZr(r7rErMrPrRrUrfrkrorqrrrsrJr|r}rwrrrrrr3rrrrr5r5r5r6s   @              &             "