o gTY@sddlZddlZddlZddlmZddlmZmZmZddl Z ddl m Z m Z mZddlmZde deded ed e jf d d Zd e deded e jfddZdedefddZ d;de dededededed e jfddZdedeee jffddZdededed eded ede d e de dededefd!d"Zdededed#edede d e de dededefd$d%Zd&d'Z   de!dSdS)=N)Path)DictOptionalTuple) ModelProto TensorProto numpy_helper) OnnxModel input_ids batch_sizesequence_lengthdictionary_sizereturncCsr|jjjtjtjtjfvsJtjj |||ftj d}|jjjtjkr*t |}|S|jjjtjkr7t |}|S)a`Create input tensor based on the graph input of input_ids Args: input_ids (TensorProto): graph input of the input_ids input tensor batch_size (int): batch size sequence_length (int): sequence length dictionary_size (int): vocabulary size of dictionary Returns: np.ndarray: the input tensor created )sizedtype) type tensor_type elem_typerFLOATINT32INT64nprandomrandintint32float32int64)r r r r datar^/var/www/visachat/venv/lib/python3.10/site-packages/onnxruntime/transformers/bert_test_data.pyfake_input_ids_datas   r segment_idscCsn|jjjtjtjtjfvsJtj||ftj d}|jjjtjkr(t |}|S|jjjtjkr5t |}|S)a,Create input tensor based on the graph input of segment_ids Args: segment_ids (TensorProto): graph input of the token_type_ids input tensor batch_size (int): batch size sequence_length (int): sequence length Returns: np.ndarray: the input tensor created r) rrrrrrrrzerosrrr)r!r r rrrrfake_segment_ids_data2s    r$max_sequence_lengthaverage_sequence_lengthcCsH|dkr||ks Jd||krtd|||Stdd|dS)N)rr)r%r&rrrget_random_lengthMs r)r( input_maskrandom_sequence_length mask_typec CsJ|jjjtjtjtjfvsJ|dkr8tj|tj d}|r,t |D] }t ||||<q!nt |D]}|||<q0n|dkr}tj ||ftj d}|rbt |D]}t ||}t |D]} d||| f<qWqLntj||ftj d} | |d| j dd| j df<n|dksJtj |ddtj d}|rt |D] }t ||||<qt |dD]6}|dkr|||d||dnd|||<|dkr|||d||dnd|d|d|<qn(t |D]}|||<qt |dD]}|||||<|||d|d|<q|jjjtjkrt|}|S|jjjtjkr#t|}|S)a"Create input tensor based on the graph input of segment_ids. Args: input_mask (TensorProto): graph input of the attention mask input tensor batch_size (int): batch size sequence_length (int): sequence length average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type (int): mask type - 1: mask index (sequence length excluding paddings). Shape is (batch_size). 2: 2D attention mask. Shape is (batch_size, sequence_length). 3: key len, cumulated lengths of query and key. Shape is (3 * batch_size + 2). Returns: np.ndarray: the input tensor created r'r"r(Nr)rrrrrrrronesrranger)r#shaperr) r*r r r&r+r,riactual_seq_lenjtemprrrfake_input_mask_dataWsZ       "  0:    r5 directoryinputsc Cstj|s(zt|Wntytd|dYnwtd|dntd|dt|D]1\}\}}t ||}t tj |d|dd }| | Wd n1sbwYq6d S) zOutput input tensors of test data to a directory Args: directory (str): path of a directory inputs (Dict[str, np.ndarray]): map from input name to value zCreation of the directory z failedz#Successfully created the directory  zWarning: directory z$ existed. Files will be overwritten.input_.pbwbN)ospathexistsmkdirOSErrorprint enumerateitemsr from_arrayopenjoinwriteSerializeToString)r6r7indexnamertensorfilerrroutput_test_datas   rM test_casesverbose random_seedc Cs|dusJtj|t|g} t|D]9} t||||}|j|i}|r0t|||||j<|r>t|||| | | ||j<|rKt| dkrKt d|| |q| S)aCreate given number of input data for testing Args: batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases dictionary_size (int): vocabulary size of dictionary for input_ids verbose (bool): print more information or not random_seed (int): random seed input_ids (TensorProto): graph input of input IDs segment_ids (TensorProto): graph input of token type IDs input_mask (TensorProto): graph input of attention mask average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type (int): mask type 1 is mask index; 2 is 2D mask; 3 is key len, cumulated lengths of query and key Returns: List[Dict[str,numpy.ndarray]]: list of test cases, where each test case is a dictionary with input name as key and a tensor as value NrzExample inputs) rrseedr/r rJr$r5lenrAappend)r r rNr rOrPr r!r*r&r+r, all_inputs _test_caseinput_1r7rrrfake_test_datas" "        rWrQc Cs:d} t|||| ||||||| | } t| |krtd| S)aCreate given number of input data for testing Args: batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases seed (int): random seed verbose (bool): print more information or not input_ids (TensorProto): graph input of input IDs segment_ids (TensorProto): graph input of token type IDs input_mask (TensorProto): graph input of attention mask average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type (int): mask type 1 is mask index; 2 is 2D mask; 3 is key len, cumulated lengths of query and key Returns: List[Dict[str,numpy.ndarray]]: list of test cases, where each test case is a dictionary with input name as key and a tensor as value i'z$Failed to create test data for test.)rWrRrA) r r rNrQrOr r!r*r&r+r,r rTrrrgenerate_test_datas$  rXcCs`|t|jkr dS|j|}||}|dur.|||}|dur.|jdkr.||jd}|S)NCastr)rRinputfind_graph_input get_parentop_type) onnx_model embed_node input_indexrZ graph_input parent_noderrrget_graph_input_from_embed_node%s   rcr^input_ids_namesegment_ids_nameinput_mask_namec Cs|}|dure||}|durtd|d}|r,||}|dur,td|d}|r@||}|dur@td|d|rEdnd|rKdnd}t||kr`td|dt||||fSt|dkrttdt||d } t| dkr| d} t|| d}t|| d}t|| d }|dur|D] } | j} d | vr| }q|durtd |||fSd}d}d}|D]} | j} d | vr| }qd | vsd| vr| }q| }q|r|r|r|||fStd)a Find graph inputs for BERT model. First, we will deduce inputs from EmbedLayerNormalization node. If not found, we will guess the meaning of graph inputs based on naming. Args: onnx_model (OnnxModel): onnx model object input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Raises: ValueError: Graph does not have input named of input_ids_name or segment_ids_name or input_mask_name ValueError: Expected graph input number does not match with specified input_ids_name, segment_ids_name and input_mask_name Returns: Tuple[Optional[np.ndarray], Optional[np.ndarray], Optional[np.ndarray]]: input tensors of input_ids, segment_ids and input_mask Nz Graph does not have input named r'rzExpect the graph to have z inputs. Got r-z'Expect the graph to have 3 inputs. Got EmbedLayerNormalizationmaskz#Failed to find attention mask inputtokensegmentz?Fail to assign 3 inputs. You might try rename the graph inputs.)'get_graph_inputs_excluding_initializersr[ ValueErrorrRget_nodes_by_op_typercrJlower) r^rdrerf graph_inputsr r!r*expected_inputs embed_nodesr_rZinput_name_lowerrrrfind_bert_inputs2sb                rt onnx_filecCsTt}t|d}||Wdn1swYt|}t||||S)aFind graph inputs for BERT model. First, we will deduce inputs from EmbedLayerNormalization node. If not found, we will guess the meaning of graph inputs based on naming. Args: onnx_file (str): onnx model path input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Returns: Tuple[Optional[np.ndarray], Optional[np.ndarray], Optional[np.ndarray]]: input tensors of input_ids, segment_ids and input_mask rbN)rrEParseFromStringreadr rt)rurdrerfmodelrLr^rrrget_bert_inputss  rzcCsJt}|jddtdd|jddtddd|jd dtd d d|jd dtd dd|jddtddd|jddtddd|jddtddd|jddtd dd|jddtddd|jddddd|jdd|jdddd d|jdd!|jd"d#d$td%d&|jd'd(ddd)d|jdd*|jd+dtd,d-d|}|S).Nz--modelTzbert onnx model path.)requiredrhelpz --output_dirFz4output test data path. Default is current directory.)r{rdefaultr|z --batch_sizer'zbatch size of inputz--sequence_lengthz maximum sequence length of inputz--input_ids_namezinput name for input idsz--segment_ids_namezinput name for segment idsz--input_mask_namezinput name for attention maskz --samplesz$number of test cases to be generatedz--seedr-z random seedz --verbose store_truezprint verbose information)r{actionr|)rOz--only_input_tensorsz-only save input tensors and no output tensors)only_input_tensorsz-az--average_sequence_lengthz)average sequence length excluding padding)r}rr|z-rz--random_sequence_lengthz3use uniform random instead of fixed sequence length)r+z --mask_typer(z^mask type: (1: mask index, 2: raw 2D mask, 3: key lengths, cumulated lengths of query and key))argparseArgumentParser add_argumentstrint set_defaults parse_args)parserargsrrrparse_argumentss   rry output_dirrc CsVt|||| \}}}t||||||||| | | }t|D]\}}tj|dt|}t||q| r5dSddl}d| vrCddgndg}|j ||d}dd| D}t|D]N\}}tj|dt|}| ||}t|D]4\}}t t|||}ttj|d |d d }||Wdn1swYqsqZdS) aICreate test data for a model, and save test data to a directory. Args: model (str): path of ONNX bert model output_dir (str): output directory batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases seed (int): random seed verbose (bool): whether print more information input_ids_name (str): graph input name of input_ids segment_ids_name (str): graph input name of segment_ids input_mask_name (str): graph input name of input_mask only_input_tensors (bool): only save input tensors, average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type(int): mask type test_data_set_NrCUDAExecutionProviderCPUExecutionProvider) providerscSsg|]}|jqSr)rJ).0outputrrr Osz-create_and_save_test_data..output_r:r;)rzrXrBr<r=rFrrM onnxruntimeget_available_providersInferenceSession get_outputsrunrrDrasarrayrErGrH)ryrr r rNrQrOrdrerfrr&r+r,r r!r*rTr1r7r6rrsession output_namesresult output_name tensor_resultrLrrrcreate_and_save_test_datasJ"   rcCst}|jdkr |j|_|j}|dur(t|j}tj|j d|j d|j}|dur8t|}|j dddnt dt |j||j |j|j|j|j|j|j|j|j|j|j|jt d|dS)Nrbatch__seq_T)parentsexist_okz7Directory existed. test data files will be overwritten.z Test data is saved to directory:)rr&r rrryr<r=rFparentr r?rArsamplesrQrOrdrerfrr+r,)rrpr=rrrmainZs8   r__main__)r()NNN)#rr<rpathlibrtypingrrrnumpyronnxrrrr^r rndarrayr r$r)boolr5rrMrWrXrcrtrzrrr__name__rrrrs*    I    :   4 ^ d      L '