o MIhvu@sddlmZddlmZddlmZddlmZddlZddl Z ddl m Z m Z m Z ddlmZeGdd d ZGd d d eZGd d d eZGdddZedkrededdZedededdZgdZdZdZededeedeedeedeejeeeed Zed!red"ed#Zed$ed%ed&ed'ed(ed)ed*ed+Zed,Zed-ed.ed/ed0ed1d2ed3ed1d4ed5ed1d6ed7ed8ed9ed:d;ed<ed=ed>d?d@dAdBd@dCdDd@dEdFd@dGdHd@dI ed=e edJddKdLD]+\Z!Z"ee!d?d@e"dMdBd@e"dNdOd@e"dPdFd@e"dQdHd@e"dRdSd; qCedTedUe edJddVdLD];\Z!Z"edWe!dXe"dMdYe"dNdSdZed[e"dQed\e"d]dSd^e"dRdSd;ed)e"d*qdSed_e#d`daedbe#dcdddS)e)OllamaPromptTemplate)BaseOutputParser)LLMChainN)DictListOptional) dataclassc@s&eZdZUeed<eed<eed<dS) SkillDatanamecountlevelN)__name__ __module__ __qualname__str__annotations__intrrN/var/www/eduai.edurigo.com/doc_train/edurigo_ai/TNA/testing/tna_skill_count.pyr s  r c@4eZdZdZdedefddZdedefddZdS) DepartmentOutputParserz-Parser for Step 1 - Department Identificationtextreturnc Cslztd|tj}|rt|WS||WSty5}zdddt|dWYd}~Sd}~ww)N\{.*?\}Unknownlowz Parse error: identified_department confidence reasoning) researchDOTALLjsonloadsgroup_extract_department_from_text Exceptionr)selfr json_matcherrrparses  zDepartmentOutputParser.parsecCs ddddS)Nrmediumz#Could not parse structured responserr)r*rrrrr("sz4DepartmentOutputParser._extract_department_from_textN)rrr__doc__rrr-r(rrrrrsrc@r) SkillAnalysisOutputParserz+Parser for Step 2 - Skill Priority Analysisrrc Csz?td|tj}|rt|}d|dWStd|tj}|r:t|}dt|tr5|gdWS|dWS||WSt yY}zdt |gdWYd}~Sd}~ww)Nz\[.*?\]T)successskill_recommendationsrFr1errorr2) r"r#r$r%r&r' isinstancedict_extract_skills_from_textr)r)r*rr+ skills_data skill_datar,rrrr-,s. zSkillAnalysisOutputParser.parsec sg}|d}i}|D]rdrq td}|r8|r.d|vr.||d|di}q tfdddDrc|rPd|vrP||i}td d d d }||d<q d  vr{t d}|rzt |d|d<q tfdddDrd d tfdddDr|d<q d vrt d}|rt |d|d<q |rd|vr|||D]&} | dd| dd| dd| dd| dd| ddq|rd nd!|t |d"S)#z,Enhanced text extraction for multiple skills #z ^\d+\.\s*(.+) skill_namec3|] }|vVqdSNlower.0keywordlinerr _zFSkillAnalysisOutputParser._extract_skills_from_text..)zskill:z skill_name:- z^[\-\*\d\.\s]*:priorityz(\d+(?:\.\d+)?)priority_percentagec3r>r?r@rBrErrrGlrH)zlevel:requiredc3r>r?r@)rCr) level_textrrrGnrH)beginner intermediateexpertrequired_levelcoveragetarget_coveragegI@ Intermediatecurrent_coverageggN@gap_typez Missing Skillr!z)Standard skill requirement for departmentTF)r1r2extracted_count)stripsplit startswithr"matchappendr'anysubrAr#float setdefaultlen) r*rskillslines current_skillnumbered_matchr<priority_matchcoverage_matchskillr)rPrFrr7Is^                  z3SkillAnalysisOutputParser._extract_skills_from_textN)rrrr/rrr-r7rrrrr0)sr0c @s*eZdZd*dedefddZdefdd Zdefd d Zd ed eede fddZ dede dee de fddZ d ed eede dede f ddZdedeedeefddZdee de defddZdedee fdd Zd!edefd"d#Zd$ee de dee fd%d&Zd$ee de fd'd(Zd)S)+TwoStepTNASystem gemma3:12bhttp://localhost:11434 model_namebase_urlcCsf||_||_t||ddd|_t|_t|_t|j| |jdd|_ t|j| |jdd|_ dS)z Initialize Two-Step TNA System Args: model_name: Ollama model name base_url: Ollama server URL g?i)modelrp temperature num_predictF)llmprompt output_parserverboseN) rorprrtr dept_parserr0 skill_parserr_create_department_prompt dept_chain_create_skill_analysis_prompt skill_chain)r*rorprrr__init__s, zTwoStepTNASystem.__init__rcCsd}tddg|dS)z=Create prompt template for Step 1 - Department Identificationas You are an expert at identifying organizational departments based on user descriptions. TASK: Analyze the user prompt and identify which specific department they are referring to from the organization's department list. ORGANIZATION DEPARTMENTS: {organization_departments} USER PROMPT: "{user_prompt}" INSTRUCTIONS: 1. Carefully analyze keywords, responsibilities, and context in the user prompt 2. Match the description to the most appropriate department from the provided list 3. Consider typical department functions and responsibilities 4. Choose the best match from the given department list 5. Provide confidence level and clear reasoning RESPOND IN THIS EXACT JSON FORMAT: {{ "identified_department": "exact department name from the list", "confidence": "high/medium/low", "reasoning": "brief explanation of why this department was chosen" }} JSON Response: user_promptorganization_departmentsinput_variablestemplaterr*rrrrrzs z*TwoStepTNASystem._create_department_promptcCsd}tgd|dS)z;Create prompt template for Step 2 - Skill Priority Analysisa You are an expert Training Need Analysis (TNA) consultant specializing in skill gap analysis. TASK: Analyze the current skills for the {department} department and recommend skill priorities that need to be addressed. DEPARTMENT: {department} TOTAL EMPLOYEES: {employee_count} CURRENT SKILLS: {current_skills} ANALYSIS REQUIREMENTS: 1. COVERAGE ANALYSIS: - Calculate current skill coverage percentages - Identify skills with insufficient coverage - Standard target: 50-70% coverage for most skills, 70%+ for critical skills 2. LEVEL ANALYSIS: - Assess if current skill levels are appropriate for the department - Consider if Beginner/Intermediate/Expert levels match job requirements 3. MISSING SKILLS: - Identify essential skills that are completely missing - Consider both technical and soft skills needed for this department 4. PRIORITY CALCULATION: - Base priority on: coverage gaps, level gaps, business impact, urgency - Priority scale: 90-100% (Critical), 70-89% (Important), 50-69% (Moderate), 30-49% (Nice-to-have), 10-29% (Low) 5. DEPARTMENT-SPECIFIC FOCUS: - Consider the core functions and responsibilities of {department} - Include industry-standard skills for this department type - Think about both current needs and future requirements IMPORTANT: You MUST provide EXACTLY 10-15 skill recommendations. Include: - Analysis of existing skills that need improvement - Essential missing skills for this department - Both technical and soft skills - Skills at different priority levels RESPOND WITH JSON ARRAY OF EXACTLY 10-15 SKILL RECOMMENDATIONS: [ {{ "skill_name": "exact skill name", "required_level": "Beginner/Intermediate/Expert", "priority_percentage": 85.5, "current_coverage": 25.0, "target_coverage": 70.0, "gap_type": "Coverage Gap/Level Gap/Missing Skill/Enhancement", "reasoning": "detailed explanation of why this skill needs attention and the assigned priority" }}, {{ "skill_name": "another skill name", "required_level": "Intermediate", "priority_percentage": 78.0, "current_coverage": 0.0, "target_coverage": 60.0, "gap_type": "Missing Skill", "reasoning": "explanation for this skill priority" }} ] CRITICAL: Provide MINIMUM 10 skills, MAXIMUM 15 skills. Do not provide fewer than 10 recommendations. Sort by priority percentage (highest first). Include a variety of skills: existing skills needing improvement + completely missing essential skills. JSON Response: departmentemployee_countcurrent_skillsrrrrrrr|s Dz.TwoStepTNASystem._create_skill_analysis_promptrrc Cszctdtd|td||d|d}|j|}t|tr]|dd}|||}|rN||d<d|d <td |d |d d d|WSd|d <d|d|d<|WSddddWSty}}zdt |ddWYd}~Sd}~ww)a[ STEP 1: Use LLM to identify department from user prompt Args: user_prompt: User's description/prompt about the department organization_departments: List of departments in the organization Returns: Dict with identified department, confidence, and reasoning u2🔄 STEP 1: Identifying department from prompt...z Prompt: z Available Departments: z, )rrrrJTr1u!✅ STEP 1 Complete: Identified 'z' with r unknownz confidenceFz Department 'z ' not found in organization listr4z2Failed to parse department identification responseN)r1r4r) printjoinr{runr5r6get_match_department_to_listr)r)r*rr chain_inputresultidentified_dept matched_deptr,rrrstep1_identify_departments<     z*TwoStepTNASystem.step1_identify_departmentrrrc Csztd|dtd|tdt|d|||}|||d}|j|}tdt|tdt|d d d t|tr| d rt| d g}td|d| |d |}d||t||| |t|j |t|dd} tdt|d| WStdt|} |j | } | d r| d r| | d |}tdt|dd||t||| |d| ddddWSddgdtt|dkrt|d dd nt|idWSty} zdt| gd WYd } ~ Sd } ~ ww)!a STEP 2: Use LLM to analyze skills for the identified department Args: department: Department identified from Step 1 employee_count: Total number of employees in department current_skills: List of current skills with counts and levels Returns: Dict with skill priority recommendations u"🔄 STEP 2: Analyzing skills for z department... Employee Count:  Current Skills: z skills providedru 🔍 Debug - LLM Response type: u🔍 Debug - Response preview: Nz...r1r2u🔍 Debug - Parsed z skills from JSON responseT)raw_response_typeraw_skill_countprocessed_skill_count)r1rrtotal_recommendationsr2analysis_summary debug_infou✅ STEP 2 Complete: Generated z skill recommendationsuY⚠️ Warning: Received non-dict or unsuccessful response, attempting manual parsing...u0✅ STEP 2 Complete (manual parsing): Generated manual_extractionrZr)parsing_methodrZFzIFailed to parse skill analysis response with both JSON and manual methods raw_responsei)r1r4r2rr3)rrd_format_skills_for_llmr}rtyperr5r6r_post_process_recommendations_generate_summaryrryr7r)) r*rrrskills_descriptionrr skill_countprocessed_recs final_resultraw_text manual_resultr,rrrstep2_analyze_skillsSs|     , z%TwoStepTNASystem.step2_analyze_skills skills_inputc Cstdtd|||}|dsdd|dd|dS|d }||}||||}|ds?dd |dd ||d Sd |||d|ddS)a Complete two-step TNA analysis Args: user_prompt: User's description of department/team organization_departments: List of organization's departments employee_count: Total employees in the department skills_input: Current skills data (formatted string) Returns: Complete analysis results from both steps u&🚀 Starting Complete TNA Analysis...zP================================================================================r1Fr=r4z Department identification failed)r1stepr4 step1_resultrzSkill analysis failed)r1rr4r step2_resultTr2r)r1rrfinal_recommendationsr)rrr_parse_skill_inputr) r*rrrrrrrrrrrcomplete_tna_analysiss2       z&TwoStepTNASystem.complete_tna_analysisr dept_listcCs`|sdS|}|D] }||kr|Sq |D]}||vs)||vr-|SqdS)z?Match identified department with organization's department listN)rAr[)r*rridentified_lowerdeptrrrrs  z*TwoStepTNASystem._match_department_to_listrecCs`|sdSg}|D]"}|j|d}|d|jd|jd|d|dd|jd qd |S) z#Format current skills for LLM inputz No current skills data provided.drIz: /z employees (.1fz% coverage) at z levelr:)r r_r rr)r*rer formattedrkrUrrrrs* z'TwoStepTNASystem._format_skills_for_llmc Csg}|s|Std|}|D]-}td|}|r?|d}t|d}||d}|t|||q|S)z/Parse skill input string into SkillData objectsz,\s*z(.+?)-(\d+)\s*\((.+?)\)r=r) r[r"r\r^r'r_normalize_skill_levelr_r ) r*rre skill_partspartr^r<r rrrrrsz#TwoStepTNASystem._parse_skill_inputrcCsH|}|dr dS|dsd|vrdS|ds |dr"dSdS) z(Normalize skill level to standard formatbeginBeginnerinterr.rWrSadvExpert)rAr])r*r level_lowerrrrr#s z'TwoStepTNASystem._normalize_skill_levelrecommendationscCsg}|D]g}zZ|dd||ddtdtdt|ddtdtdt|d dtdtdt|d d|d d|d d d}t|d d||d<||Wqtt fykYqw|j dddd|S)z)Post-process and validate recommendationsr<rrTrWrrrN2rXrVrYr!zNo reasoning provided)r<rTrNrXrVrYr!target_employee_countcSs|dS)NrNr)xrrrIsz@TwoStepTNASystem._post_process_recommendations..T)keyreverse) rr[rmaxminrbrr_ ValueError TypeErrorsort)r*rr processedrec processed_recrrrr.s( z.TwoStepTNASystem._post_process_recommendationscCs|sddiStdd|D}tdd|D}tdd|D}tdd|Dt|}t||||d t|d |rD|d d dSd dS)zGenerate analysis summarymessagezNo recommendations generatedcSsg|] }|ddkr|qS)rNPrrCrrrr Qz6TwoStepTNASystem._generate_summary..cSs,g|]}d|dkrdkrnn|qS)rrNrrrrrrrRs,cSsg|] }|ddkr|qS)rNrrrrrrrSrcss|]}|dVqdS)rNNrrrrrrGUsz5TwoStepTNASystem._generate_summary..) high_prioritymedium_priority low_priorityr=rr<N) total_skillspriority_distributionaverage_prioritytop_priority_skill)rdsumround)r*rrrr avg_priorityrrrrLs z"TwoStepTNASystem._generate_summaryN)rmrn)rrrrr~rrzr|rrrrr rrr rrrrrrrrrrrls.'!L 5 f 6 rl__main__z#Initializing Two-Step TNA System...rm)roze ====================================================================================================z'EXAMPLE: COMPLETE TWO-STEP TNA ANALYSISzd====================================================================================================zZWe need training for our team that handles customer complaints and manages support tickets)Sales MarketingzHuman ResourceszInformation TechnologyFinancezCustomer Service OperationszxCommunication-12(Intermediate), Active Listening-6(Intermediate), CRM Tools-5(Beginner), Problem Solving-8(Intermediate)u📝 Input Data:z User Prompt: z Organization Departments: rr)rrrrr1u# 🎉 COMPLETE ANALYSIS SUCCESSFUL!ru 📍 STEP 1 RESULTS:z Identified Department: rz Confidence: r z Reasoning: r!rru 📊 STEP 2 RESULTS SUMMARY:z Total Recommendations: rz High Priority Skills: rrz Medium Priority Skills: rz Low Priority Skills: rz Top Priority Skill: rz Average Priority: r%u# 🎯 TOP 10 SKILL RECOMMENDATIONS:zx------------------------------------------------------------------------------------------------------------------------Rankz<4 z Skill Namez<25z Priority %z<10zRequired Levelz<15zGap Typez<20zTarget Coverager r=r<rNz<10.1frTrYrVru# 💡 DETAILED REASONING FOR TOP 3:zP--------------------------------------------------------------------------------rr:z. z (z % priority)z Gap Type: z Current Coverage: rXu% → Target: u❌ Analysis Failed at Step rrz Error: r4z Unknown error)$langchain_community.llmsrlangchain.promptsrlangchain.schemarlangchain.chainsrr%r"typingrrr dataclassesr r rr0rlrr tna_systemrrrrrrstep1step2summary enumerateirrrrrrs    ` ]  4"&"