增强 Whisper 转录：预处理和后处理技术

此 notebook 提供了一个改进 Whisper 转录的指南。我们将通过修剪和分割来简化您的音频数据，从而提高 Whisper 的转录质量。转录后，我们将通过添加标点符号、调整产品术语（例如，“five two nine”到“529”）和缓解 Unicode 问题来改进输出。这些策略将有助于提高您的转录清晰度，但请记住，根据您的独特用例进行自定义可能会有所帮助。

设置

首先，让我们导入一些不同的库

PyDub 是一个简单易用的 Python 库，用于音频处理任务，例如切片、连接和导出音频文件。
IPython.display 模块中的 Audio 类允许您创建一个音频控件，该控件可以在 Jupyter notebook 中播放声音，从而提供一种直接在 notebook 中播放音频数据的简单方法。
对于我们的音频文件，我们将使用由 ChatGPT 编写并由作者朗读的虚构的财报电话会议。此音频文件相对较短，但希望为您提供一个说明性的想法，说明如何将这些预处理和后处理步骤应用于任何音频文件。

# set download paths earnings_call_remote_filepath = "https://cdn.openai.com/API/examples/data/EarningsCall.wav" # set local save locations earnings_call_filepath = "data/EarningsCall.wav" # download example audio files and save locally ssl._create_default_https_context = ssl._create_unverified_context urllib.request.urlretrieve(earnings_call_remote_filepath, earnings_call_filepath)

有时，开头有长时间静音的文件可能会导致 Whisper 错误地转录音频。我们将使用 Pydub 来检测和修剪静音。

这里，我们将分贝阈值设置为 20。如果您愿意，可以更改此值。

# Function to detect leading silence # Returns the number of milliseconds until the first sound (chunk averaging more than X decibels) def milliseconds_until_sound(sound, silence_threshold_in_decibels=-20.0, chunk_size=10): trim_ms = 0 # ms assert chunk_size > 0 # to avoid infinite loop while sound[trim_ms:trim_ms+chunk_size].dBFS < silence_threshold_in_decibels and trim_ms < len(sound): trim_ms += chunk_size return trim_ms

def trim_start(filepath): path = Path(filepath) directory = path.parent filename = path.name audio = AudioSegment.from_file(filepath, format="wav") start_trim = milliseconds_until_sound(audio) trimmed = audio[start_trim:] new_filename = directory / f"trimmed_{filename}" trimmed.export(new_filename, format="wav") return trimmed, new_filename

def transcribe_audio(file,output_dir): audio_path = os.path.join(output_dir, file) with open(audio_path, 'rb') as audio_data: transcription = client.audio.transcriptions.create( model="whisper-1", file=audio_data) return transcription.text

有时，我们会在转录中看到 unicode 字符注入，删除任何非 ASCII 字符应该有助于缓解此问题。

请记住，如果您要转录希腊语、西里尔语、阿拉伯语、中文等，则不应使用此功能

此功能将为我们的转录添加格式和标点符号。 Whisper 生成的转录带有标点符号，但没有格式。

# Define function to add punctuation def punctuation_assistant(ascii_transcript): system_prompt = """You are a helpful assistant that adds punctuation to text. Preserve the original words and only insert necessary punctuation such as periods, commas, capialization, symbols like dollar sings or percentage signs, and formatting. Use only the context provided. If there is no context provided say, 'No context provided'\n""" response = client.chat.completions.create( model="gpt-3.5-turbo", temperature=0, messages=[ { "role": "system", "content": system_prompt }, { "role": "user", "content": ascii_transcript } ] ) return response

我们的音频文件是来自虚假财报电话会议的录音，其中包含许多金融产品。此功能可以帮助确保如果 Whisper 错误地转录了这些金融产品名称，则可以对其进行更正。

# Define function to fix product mispellings def product_assistant(ascii_transcript): system_prompt = """You are an intelligent assistant specializing in financial products; your task is to process transcripts of earnings calls, ensuring that all references to financial products and common financial terms are in the correct format. For each financial product or common term that is typically abbreviated as an acronym, the full term should be spelled out followed by the acronym in parentheses. For example, '401k' should be transformed to '401(k) retirement savings plan', 'HSA' should be transformed to 'Health Savings Account (HSA)' , 'ROA' should be transformed to 'Return on Assets (ROA)', 'VaR' should be transformed to 'Value at Risk (VaR)' , and 'PB' should be transformed to 'Price to Book (PB) ratio'. Similarly, transform spoken numbers representing financial products into their numeric representations, followed by the full name of the product in parentheses. For instance, 'five two nine' to '529 (Education Savings Plan)' and 'four zero one k' to '401(k) (Retirement Savings Plan)'. However, be aware that some acronyms can have different meanings based on the context (e.g., 'LTV' can stand for 'Loan to Value' or 'Lifetime Value'). You will need to discern from the context which term is being referred to and apply the appropriate transformation. In cases where numerical figures or metrics are spelled out but do not represent specific financial products (like 'twenty three percent'), these should be left as is. Your role is to analyze and adjust financial product terminology in the text. Once you've done that, produce the adjusted transcript and a list of the words you've changed""" response = client.chat.completions.create( model="gpt-4", temperature=0, messages=[ { "role": "system", "content": system_prompt }, { "role": "user", "content": ascii_transcript } ] ) return response

此功能将创建一个新文件，并在原始文件名后附加“trimmed”

我们的虚假财报音频文件长度相当短，因此我们将相应地调整分段。请记住，您可以根据需要调整分段长度。

# Segment audio trimmed_audio = AudioSegment.from_wav(trimmed_filename) # Load the trimmed audio file one_minute = 1 * 60 * 1000 # Duration for each segment (in milliseconds) start_time = 0 # Start time for the first segment i = 0 # Index for naming the segmented files output_dir_trimmed = "trimmed_earnings_directory" # Output directory for the segmented files if not os.path.isdir(output_dir_trimmed): # Create the output directory if it does not exist os.makedirs(output_dir_trimmed) while start_time < len(trimmed_audio): # Loop over the trimmed audio file segment = trimmed_audio[start_time:start_time + one_minute] # Extract a segment segment.export(os.path.join(output_dir_trimmed, f"trimmed_{i:02d}.wav"), format="wav") # Save the segment start_time += one_minute # Update the start time for the next segment i += 1 # Increment the index for naming the next file

# Get list of trimmed and segmented audio files and sort them numerically audio_files = sorted( (f for f in os.listdir(output_dir_trimmed) if f.endswith(".wav")), key=lambda f: int(''.join(filter(str.isdigit, f))) )

Good afternoon, everyone. And welcome to FinTech Plus Sync's second quarter 2023 earnings call. I'm John Doe, CEO of FinTech Plus. We've had a stellar Q2 with a revenue of 125 million, a 25% increase year over year. Our gross profit margin stands at a solid 58%, due in part to cost efficiencies gained from our scalable business model. Our EBITDA has surged to 37.5 million, translating to a remarkable 30% EBITDA margin. Our net income for the quarter rose to 16 million, which is a noteworthy increase from 10 million in Q2 2022. Our total addressable market has grown substantially thanks to the expansion of our high yield savings product line and the new RoboAdvisor platform. We've been diversifying our asset-backed securities portfolio, investing heavily in collateralized. debt obligations, and residential mortgage-backed securities. We've also invested $25 million in AAA rated corporate bonds, enhancing our risk adjusted returns. As for our balance sheet, total assets reached $1.5 billion with total liabilities at $900 million, leaving us with a solid equity base of $600 million. Our debt-to-equity ratio stands at 1.5, a healthy figure considering our expansionary phase. We continue to see substantial organic user growth, with customer acquisition cost dropping by 15% and lifetime value growing by 25%. Our LTVCAC ratio is at an impressive 3.5%. In terms of risk management, we have a value-at-risk model in place with a 99%... confidence level indicating that our maximum loss will not exceed 5 million in the next trading day. We've adopted a conservative approach to managing our leverage and have a healthy tier one capital ratio of 12.5%. Our forecast for the coming quarter is positive. We expect revenue to be around 135 million and 8% quarter over quarter growth driven primarily by our cutting edge blockchain solutions and AI driven predictive analytics. We're also excited about the upcoming IPO of our FinTech subsidiary Pay Plus, which we expect to raise 200 million, significantly bolstering our liquidity and paving the way for aggressive growth strategies. We thank our shareholders for their continued faith in us and we look forward to an even more successful Q3. Thank you so much.

Good afternoon, everyone. And welcome to FinTech Plus Sync's second quarter 2023 earnings call. I'm John Doe, CEO of FinTech Plus. We've had a stellar Q2 with a revenue of $125 million, a 25% increase year over year. Our gross profit margin stands at a solid 58%, due in part to cost efficiencies gained from our scalable business model. Our EBITDA has surged to $37.5 million, translating to a remarkable 30% EBITDA margin. Our net income for the quarter rose to $16 million, which is a noteworthy increase from $10 million in Q2 2022. Our total addressable market has grown substantially thanks to the expansion of our high yield savings product line and the new RoboAdvisor platform. We've been diversifying our asset-backed securities portfolio, investing heavily in collateralized debt obligations, and residential mortgage-backed securities. We've also invested $25 million in AAA rated corporate bonds, enhancing our risk-adjusted returns. As for our balance sheet, total assets reached $1.5 billion with total liabilities at $900 million, leaving us with a solid equity base of $600 million. Our debt-to-equity ratio stands at 1.5, a healthy figure considering our expansionary phase. We continue to see substantial organic user growth, with customer acquisition cost dropping by 15% and lifetime value growing by 25%. Our LTVCAC ratio is at an impressive 3.5%. In terms of risk management, we have a value-at-risk model in place with a 99% confidence level indicating that our maximum loss will not exceed $5 million in the next trading day. We've adopted a conservative approach to managing our leverage and have a healthy tier one capital ratio of 12.5%. Our forecast for the coming quarter is positive. We expect revenue to be around $135 million and 8% quarter over quarter growth driven primarily by our cutting-edge blockchain solutions and AI-driven predictive analytics. We're also excited about the upcoming IPO of our FinTech subsidiary Pay Plus, which we expect to raise $200 million, significantly bolstering our liquidity and paving the way for aggressive growth strategies. We thank our shareholders for their continued faith in us and we look forward to an even more successful Q3. Thank you so much.

Good afternoon, everyone. And welcome to FinTech Plus Sync's second quarter 2023 earnings call. I'm John Doe, CEO of FinTech Plus. We've had a stellar second quarter (Q2) with a revenue of $125 million, a 25% increase year over year. Our gross profit margin stands at a solid 58%, due in part to cost efficiencies gained from our scalable business model. Our Earnings Before Interest, Taxes, Depreciation, and Amortization (EBITDA) has surged to $37.5 million, translating to a remarkable 30% EBITDA margin. Our net income for the quarter rose to $16 million, which is a noteworthy increase from $10 million in second quarter (Q2) 2022. Our total addressable market has grown substantially thanks to the expansion of our high yield savings product line and the new RoboAdvisor platform. We've been diversifying our asset-backed securities portfolio, investing heavily in Collateralized Debt Obligations (CDOs), and Residential Mortgage-Backed Securities (RMBS). We've also invested $25 million in AAA rated corporate bonds, enhancing our risk-adjusted returns. As for our balance sheet, total assets reached $1.5 billion with total liabilities at $900 million, leaving us with a solid equity base of $600 million. Our Debt-to-Equity (D/E) ratio stands at 1.5, a healthy figure considering our expansionary phase. We continue to see substantial organic user growth, with Customer Acquisition Cost (CAC) dropping by 15% and Lifetime Value (LTV) growing by 25%. Our LTV to CAC (LTVCAC) ratio is at an impressive 3.5%. In terms of risk management, we have a Value at Risk (VaR) model in place with a 99% confidence level indicating that our maximum loss will not exceed $5 million in the next trading day. We've adopted a conservative approach to managing our leverage and have a healthy Tier 1 Capital ratio of 12.5%. Our forecast for the coming quarter is positive. We expect revenue to be around $135 million and 8% quarter over quarter growth driven primarily by our cutting-edge blockchain solutions and AI-driven predictive analytics. We're also excited about the upcoming Initial Public Offering (IPO) of our FinTech subsidiary Pay Plus, which we expect to raise $200 million, significantly bolstering our liquidity and paving the way for aggressive growth strategies. We thank our shareholders for their continued faith in us and we look forward to an even more successful third quarter (Q3). Thank you so much. Words Changed: 1. Q2 -> second quarter (Q2) 2. EBITDA -> Earnings Before Interest, Taxes, Depreciation, and Amortization (EBITDA) 3. Q2 2022 -> second quarter (Q2) 2022 4. CDOs -> Collateralized Debt Obligations (CDOs) 5. RMBS -> Residential Mortgage-Backed Securities (RMBS) 6. D/E -> Debt-to-Equity (D/E) 7. CAC -> Customer Acquisition Cost (CAC) 8. LTV -> Lifetime Value (LTV) 9. LTVCAC -> LTV to CAC (LTVCAC) 10. VaR -> Value at Risk (VaR) 11. IPO -> Initial Public Offering (IPO) 12. Q3 -> third quarter (Q3)