o AhJ{@sddlZddlZddlZddlmZmZddlmZmZm Z m Z m Z ddl m Z ddlZddlmZeGdddZeGdd d ZeGd d d ZGd d d ZGdddZddZedkrhedSdS)N) dataclassasdict)ListDictAnyOptionalTuple)Counter)Pathc@s^eZdZUdZeed<eed<eeed<eeed<eeed<ee e fed<eed<d S) SlideContentz&Structure for individual slide content slide_numbertitle main_pointssupporting_details key_quotestimestamp_range slide_typeN) __name__ __module__ __qualname____doc__int__annotations__strrrfloatrrW/var/www/eduai.edurigo.com/doc_train/edurigo_ai/Puru/youtube_slides/content_analyzer.pyr s     r c@seZdZUdZeed<eed<eed<eeed<eeed<eed<e ed<eed <e ee fed <e ed <d Z eeed <d S)AnalyzedContentz4Complete analyzed content ready for slide generationvideo_id video_title main_topic key_themesslides total_slidesestimated_presentation_timecontent_summaryanalysis_metadatasuccessN error_message)rrrrrrrr rrrrboolr(rrrrrrs   rc@sveZdZUdZeed<eed<eed<eed<eed<eed<dZ eed <dZ eed <dZ eed <d Z e eed <d S)SavedTranscriptz/Structure to match your saved transcript formatr transcriptdurationlanguage word_countr'rextraction_methodextraction_timestampNr()rrrrrrrrr)rr0r1r(rrrrrr*'s    r*c@sjeZdZdZd6deededefdd Zd ed efd d Zd ed e efddZ d7d eded e efddZ d8d eded efddZ d9d eded e efddZ deded efddZd e ed!ed e e efd"d#Zd$e ed%ed&ed efd'd(Zd:d$e ed*ed e efd+d,Zd$e ed-e ed e efd.d/Zd;d$e ed1ed e efd2d3Zd efd4d5ZdS)<YouTubeContentAnalyzerzK Analyzes extracted content and structures it for slide generation N target_slides min_slides max_slidescCs ||_||_||_hd|_dS)N>Iaianatbebydoheinisitmemyofonortoupusweandarebutcandidforhadhasherhimhisitsmayourshethewasyoubeendoesfromhaveintomustthatthemtheythisuponwerewillwithyouraboutaboveafteramongbeingbelowcouldmighttheirthesethosewouldbeforeduringshouldbetweendespitethroughtowards throughout)r5r6r7 stop_words)selfr5r6r7rrr__init__;szYouTubeContentAnalyzer.__init__textreturncCs||sdStdd|}tdd|}tdd|}tdd|}tjdd|tjd}tdd |}td d |}|}|S) zClean and normalize textr/u[♪♫🎵🎶]z\[.*?\]z\(.*?\)z<\d+:\d+:\d+\.\d+>zKind:\s*\w+\s*Language:\s*\w+flagsz\b\d+\b z\s+)resub IGNORECASEstrip)rrrrr clean_textKsz!YouTubeContentAnalyzer.clean_textcCs|sgS||}td|}g}|D]}|}t|dkr+|ds+||qtdt|dt|d|rKtd|dd d d |S) zExtract sentences from textz[.!?]+ )zkind:z language:zDebug: Extracted z sentences from characterszDebug: First sentence: rNd...) rrsplitrlenlower startswithappendprint)rrr sentencesclean_sentencessentencerrrextract_sentencesds   z(YouTubeContentAnalyzer.extract_sentencesrtop_ncs||}td|}fdd|D}t|}dd||dD}g}|} tt| dD]a} | | d| | d} t| dkr^| | j vr^| | dj vr^| | | t| dkr| | d| | dd| | d} t| d kr| | j vr| | dj vr| | q4t|} d d| |D} t t |d |d| d |d}|d |S) z(Extract key phrases and topics from text\b[a-zA-Z]{3,}\bcg|] }|jvr|qSrr.0wordrrr z>YouTubeContentAnalyzer.extract_key_phrases..cSg|]\}}|qSrr)rrcountrrrrrrcSrrr)rphraserrrrrrN) rrrfindallr most_commonrrangerrrlistset)rrrrwordsfiltered_words word_freq key_wordsphrases words_listr9r phrase_freq top_phrases key_phrasesrrrextract_key_phrases}s0   (  ( z*YouTubeContentAnalyzer.extract_key_phrasesr/rcs|rtd|}fdd|D}ng}j|dd}|r8d|dd}|r4|d |d 7}|S|rD|d }|Sd }|S) z&Identify the main topic of the contentrcrrrrrrrrrz>YouTubeContentAnalyzer.identify_main_topic..rrNr3 - rzContent Analysis)rrrrjoinr )rrr title_wordstitle_keywordsrr rrridentify_main_topics z*YouTubeContentAnalyzer.identify_main_topicr num_themesc Cs||}g}|j||dd}|d|D]/}g}|D]}||vr9|t|dkr6|dddn|q|rE|} || q|sLgd}|d|S)z$Extract main themes from the contentrrNrr) Introductionz Main Contentz Key Points Conclusion)rrrrrr ) rrrrthemesrrrelated_sentencesr theme_titlerrrextract_key_themess &  z)YouTubeContentAnalyzer.extract_key_themescontent_lengthr,cCsh|jrt|jt|j|jStdtd|d}tdtd|d}t||d}t|jt|j|S)z3Determine optimal number of slides based on contentr3r4-ir)r5maxr6minr7r)rrr, time_based word_basedoptimal_slidesrrrdetermine_slide_counts z,YouTubeContentAnalyzer.determine_slide_countr num_slidescCs|sgStdt||}g}g}t|D]*\}}||t||kr,|t|dks4|t|dkr?|r?||g}qt||krkt|dkrkt|dkr_|d|d|t||krkt|dksL|S)z'Segment content into slide-sized chunksrr)rr enumeratercopyextendpop)rrrsentences_per_slidesegmentscurrent_segmentr9rrrrsegment_contents$ , z&YouTubeContentAnalyzer.segment_contentcontent_segmentr r#cCs|sd|S|dkr dS||krdSd|dd}|j|dd }|r;|d }t|d kr9|dd d }|Sd|dS)z/Generate title for a slide based on its contentzSlide rrrrNrr3rr2/rz Key Point )rrr r)rrr r# combined_textrr rrrcreate_slide_titles   z)YouTubeContentAnalyzer.create_slide_title max_pointscCsg}|d|D]6}|}tjdd|tjd}|r>t|dkr,|d|ddn|}|ds9|d7}||q|S) z*Extract main points from a content segmentNz7^(and|but|so|then|also|furthermore|moreover|however)\s+r/rrr).!?r)rrrrrupperendswithr)rrrpointsrpointrrrextract_main_points(s,  z*YouTubeContentAnalyzer.extract_main_pointsrcCsg}t|}|D]9}||vrAt|dkrAtd|s2d|vs2d|vs2d|vs2d|vrA|}t|dkrA||q|S)z6Extract supporting details that complement main pointsr3z\d+examplezsuch as includinglike)rrrsearchrrr)rrrdetailsused_sentencesrdetailrrrextract_supporting_details@s       z1YouTubeContentAnalyzer.extract_supporting_detailsr max_quotescCstg}|D]3}t|dkr7t|dkr7d|vs%|ddks%td|r7|}||t||kr7|Sq|S)z*Extract memorable quotes or key statementssaid"rz[!.])rrrrrrr)rrrquotesrquoterrrextract_key_quotesUs    z)YouTubeContentAnalyzer.extract_key_quotesc"Csz|j}|j}|j}|j}t|dd|}td|tdt|dtd|td|dd d ||}td t|d||}|std |dd | d} dd| D} | r{tdt| d| dd}nt ||dggdddiddt|dt|d WStdt|d| ||} | |} | ||} ||| } tdt| d| dg}t| D]U\}}|d}|dkrd}n |t| krd }nd!}|||t| }||}|||}||}|t| }|d|}||}t|||||||f|d"}||qd#t|d$t|d%| d}t|d&}td'd(|Dd)}||}|t|t||r]td*d(|Dt|nd|rg|t|ndtd+jd,}t ||| | |t||||d-d. WSty} z4ddl}!td/t| td0|!t t|d1d2d3dggdddidd4t| d WYd} ~ Sd} ~ ww)5zLMain analysis function that processes extracted content into slide structurerzVideo Analysis - zDebug: Starting analysis for zDebug: Transcript length: rzDebug: Video title: zDebug: Transcript preview: Nrz Debug: Clean transcript length: z6Debug: No sentences extracted. Raw transcript sample: ircSs$g|]}t|dkr|qSr)rr)rsrrrrs$z:YouTubeContentAnalyzer.analyze_content..z$Debug: Alternative extraction found z sentencesrr/rFz*No content to analyze. Transcript length: z, Clean length: rrr r!r"r#r$r%r&r'r(zDebug: Processing zDebug: Created z content segments for z slidesrintro conclusioncontentr r rrrrrz Analysis of z sentences across z slides, focusing on g?css$|] }t|jt|jVqdSN)rrrrsliderrr s"z9YouTubeContentAnalyzer.analyze_content..g?css|]}t|jVqdSr)rrrrrrrsdatetime)original_word_countsentences_analyzedslides_createdavg_points_per_slidecontent_densityanalysis_timestampT) rrr r!r"r#r$r%r&r'zDebug: Exception occurred: zDebug: Traceback: runknownErrorzAnalysis failed: ) rr+r,r.getattrrrrrrrrrrrrrrrrr rrsum __import__r now isoformat Exception tracebackr format_exc)"rextracted_contentrr+r,r.rclean_transcriptr alt_sentencesr r!rcontent_segmentsr"r9segmentr rr rrrsegment_duration start_timeend_timerr% base_time detail_timeestimated_timer&errrranalyze_contentis              $ "    z&YouTubeContentAnalyzer.analyze_content)Nr3r4r)r/)r)r)r)rrrrrrrrrrrrrrrrrrrrrrr&rrrrr26s." " r2c @seZdZdZddedefddZded eefd d Zd e e ee ffd d Z ddedee ded eefddZded efddZded eefddZded eefddZdefddZdS)TranscriptAnalysisManagerz%Manages analysis of saved transcripts transcriptsanalysistranscripts_dir analysis_dircCs.t||_t||_|jjddt|_dS)NT)exist_ok)r r*r+mkdirr2analyzer)rr*r+rrrrs   z"TranscriptAnalysisManager.__init__rrc CszG|j|d}|std|WdSt|ddd }t|}Wdn1s.wYtd i|}td|d|j|WStyd}ztd |d|WYd}~dSd}~ww) z Load a specific saved transcriptz_transcript.jsonu❌ Transcript file not found: Nrutf-8encodingu✅ Loaded transcript for : u!❌ Error loading transcript for r) r*existsropenjsonloadr*rr)rrtranscript_filefdatar+r%rrrload_transcripts  z)TranscriptAnalysisManager.load_transcriptc Csg}|jdD]c}zEt|ddd }t|}Wdn1s"wY||dd|dd |d d |d d |d d|ddt|dWqtyk}zt d|d|WYd}~qd}~wwt |ddddS)z$List all available saved transcriptsz*_transcript.jsonr/r0r1NrrrUnknownr.rr,r'Fr0)rr r.r,r'r0 file_pathzError reading r3cSs |ddS)Nr.r)get)xrrr3s zFTranscriptAnalysisManager.list_available_transcripts..T)keyreverse) r*globr5r6r7rr>rrrsorted)rr(r=r9r:r%rrrlist_available_transcriptss(         z4TranscriptAnalysisManager.list_available_transcriptsNFr5 force_refreshcCs|j|d}|s$|r$z td|||WStdY||}|r.|js4tddStd|dt|d}||}|jrW||td ||Std |j dS) z#Analyze a specific saved transcript_analysis.jsonu#📁 Loading existing analysis for u9⚠️ Failed to load existing analysis, creating new oneu6❌ Cannot analyze: transcript not available or failedNu🔄 Analyzing transcript for rr5u✅ Analysis completed for u❌ Analysis failed: ) r+r4r load_analysisr;r'r2r& save_analysisr()rrr5rF analysis_filer+r.analyzed_contentrrranalyze_transcript5s(        z,TranscriptAnalysisManager.analyze_transcriptrLc Csz7|j|jd}t|}t|ddd}tj||dddWdn1s)wYtd |Wd StyQ}ztd |WYd}~dSd}~ww) zSave analysis results to JSONrGwr0r1rF)indent ensure_asciiNu💾 Analysis saved to: Tu❌ Failed to save analysis: )r+rrr5r6dumprr)rrLrK analysis_dictr9r%rrrrJXsz'TranscriptAnalysisManager.save_analysisc Cs0z}|j|d}|sWdSt|ddd }t|}Wdn1s'wYg}|dgD]"}t|d|d|d |d |d t|d |d d}||q4t |d|d|d|d||d|d|d|d|d|dd }|WSt y} zt d| WYd} ~ dSd} ~ ww)zLoad previously saved analysisrGNr/r0r1r"r r rrrrrrrrr r!r#r$r%r&r'r(ru❌ Error loading analysis: ) r+r4r5r6r7r>r tuplerrrr) rrrKr9r:r" slide_datarrLr%rrrrIjsL   z'TranscriptAnalysisManager.load_analysisc Cs||}|r |js dSd|jd|jd|jdd|jddd|jd d }|jD]}|d |j d |j d t |j d7}q.|S)z+Get a formatted summary of analysis resultsNu 📊 **ANALYSIS SUMMARY: u'** 🎯 **Overview:** • Main Topic: u • Total Slides: u • Key Themes: , r3u • Estimated Time: .1fu# minutes 📋 **Slide Structure:**z . z (z points)) rIr'rr r#rr!r$r"r r rr)rranalyzedsummaryrrrrget_analysis_summarys"   (z.TranscriptAnalysisManager.get_analysis_summaryc Cs||}|r |jstd|dStd|jdtdtd|jtd|jtd|jd d td d |jtd td|j D]x}td|j d|j d|j d|j rwtd|j D] }td|qm|jrtd|jD] }td|q|jrtd|jD] }td|dqt|jd}t|jd}td|dd|ddd |dd|ddqPdS)!z.Print detailed analysis with all slide contentu❌ No analysis available for Nu 🎬 **DETAILED ANALYSIS: z**z<============================================================u🎯 Main Topic: u📊 Total Slides: u⏱️ Estimated Time: rVz minutesu🔑 Key Themes: rUu 📋 **SLIDE BREAKDOWN:**z(----------------------------------------u 🎯 **Slide r3z** ()u 📌 Main Points:u • u 📝 Supporting Details:z - u 💬 Key Quotes:z "rrru ⏰ Timestamp: <:02dr)rIr'rrr r#r$rr!r"r r rrrrrr) rrrXrrrrtimestamp_start timestamp_endrrrprint_detailed_analysiss<   "   6z1TranscriptAnalysisManager.print_detailed_analysis)r(r))NF)rrrrrrrr*r;rrrrErr)rrMrJrIrZrarrrrr's$#-r'c Cs*t}tdtd|}t|dD]4\}}|drdnd}t|d|d|d d |d td |d dd|ddd|dq|sQtddSdd|D}|r|dd }td||j|dd}|rtd||td|d}|dkr||dSdSdSdS) z"Main function to demonstrate usageu📁 Available Transcripts:z(========================================rr'u✅u❌rWrrr3r z Words: r.,z | Duration: r,z.0fz s | Method: r0uJ❌ No transcripts found. Make sure you have saved some transcripts first.NcSsg|]}|dr|qS)r'r)rtrrrrszmain..ru$ 🔄 Analyzing example transcript: rH z Show detailed analysis for z ? (y/N): y) r'rrErrMrZinputrra) managerr(r9r+statussuccessful_transcriptsexample_video_idrXchoicerrrmains0&,  rm__main__)rr6os dataclassesrrtypingrrrrr collectionsr mathpathlibr r rr*r2r'rmrrrrrs0   I]'