ChatGPT Text Streaming: “Typing Effect” with OpenAI’s API and Javascript / PHP / Ajax

Have you ever wondered how to re-create ChatGPT’s famous “typing effect” on your own web applications?

Streaming data, as opposed to delivering it in a static format, can create engaging interfaces and increase user satisfaction. Because users won’t need to wait as long for a text response to appear, they stay more engaged and less impatient. Plus… it just looks cool!

In this blog post, we will explain the concept of streaming and its advantages, as well as demonstrate how to accomplish real-time text streaming using OpenAI’s ChatGPT API, and Ajax (PHP with JavaScript.)

Update: A Python code example is also now included, below.

1. What is Streaming and Its Advantages

Streaming in web applications refers to providing data to the user as a continuous feed instead of bundling it as a single static package. It is particularly useful in contexts where content might be generated on-the-fly or when the data size might be unpredictable. Some benefits of streaming include:

• Real-time visualization: Streaming data can create an immersive user experience that reflects ongoing events, like live chats or data correlations.
• User engagement: Real-time content display can keep users interested and encourage them to engage with your platform.
• Efficient data handling: Streaming allows for handling large amounts of data without causing significant delays or overloading server resources.

2. Prerequisites

To streamline real-time text streaming with OpenAI’s ChatGPT API, we will be using the following technologies:

• OpenAI API: Sign up for an OpenAI account and obtain the API key
• PHP: Ensure that your web server is set up with PHP capabilities
• (Update) Python, if you wish to try this method with Python
• JavaScript: Basic knowledge on how to interact with HTML via JavaScript

3. Setting Up the Server-Side PHP Script

The first step is to create a server-side PHP script that communicates with the OpenAI API. This script should use event streams to send data to the client in chunks, allowing for real-time display.

<?php
// Set the headers for Server-Sent Events (SSE) and prevent caching
header('Content-Type: text/event-stream');
header('Cache-Control: no-cache');
header('Connection: keep-alive');

// Define the API key (replace this with your actual API key)
define('API_KEY', 'sk-YOUR-KEY-HERE');

// Get the input from the user via a GET parameter
$input = $_GET['input'];
$prompt = $input;

// Prepare the request data to be sent to the GPT API
$data = array(
  'model' => 'gpt-3.5-turbo',
  'stream' => true,
  'messages' => array(
    array(
      'role' => 'user',
      'content' => $prompt
    )
  )
);

// Set the curl options for the API request
$options = array(
  CURLOPT_URL => 'https://api.openai.com/v1/chat/completions',
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_POSTFIELDS => json_encode($data),
  CURLOPT_HTTPHEADER => array(
    'Content-Type: application/json',
    'Authorization: Bearer ' . API_KEY
  ),
  // Process each chunk of data as it arrives
  CURLOPT_WRITEFUNCTION => function($ch, $chunk) {
    echo $chunk;
    return strlen($chunk);
  }
);

// Initialize the curl request
$ch = curl_init();

// Set the curl options specified earlier
curl_setopt_array($ch, $options);

// Execute the curl request and close the connection
curl_exec($ch);
curl_close($ch);

?>

The script uses cURL to make an API call, and the CURLOPT_WRITEFUNCTION is set to a custom function that processes data chunks as they arrive, sending the data to the client-side JavaScript using echo.

It uses Server-Sent Events (SSE) for real-time, continuous data communication while processing the API response in chunks, allowing for a live stream of data from the API.

Expected Output:

You should now be able to test the PHP file. Either set the input prompt text variable “$input” manually for testing, or feed it a $_GET variable in the URL as “?input=Your prompt here.”

You should be able to see the “data chunks” streaming, as the script parses the response from the ChatGPT API:

Python Version:

Remember, before you run this code you need to install requests module if you haven't installed it already. You can install it by running pip install requests in your command line.

import requests
import json

# Define the API key (replace this with your actual API key)
API_KEY = 'sk-YOUR-KEY-HERE'

# Get the input from the user
input = input("Please enter the input: ")

# Prepare the request data to be sent to the GPT API
data = {
  'model': 'gpt-3.5-turbo',
  'stream': True,
  'messages': [
    {
      'role': 'user',
      'content': input
    }
  ]
}

# Set the headers for the request
headers = {
  'Content-Type': 'application/json',
  'Authorization': 'Bearer ' + API_KEY
}

# Send the request to the OpenAI API and process each chunk of data as it arrives
response = requests.post('https://api.openai.com/v1/chat/completions', data=json.dumps(data), headers=headers, stream=True)

if response.status_code == 200:
    for chunk in response.iter_content(chunk_size=None):
        if chunk:
            print(chunk.decode())
else:
    print("Request failed with status code: ", response.status_code)

4. What are Server-Sent Events (SSE)?

Server-Sent Events (SSE) are a technology that allows for real-time, continuous data communication between the server and the client (usually a web browser) over a single HTTP connection. It is a simple and efficient protocol for sending real-time updates from the server to the client, primarily used to enable real-time data updates like notifications or live updates in applications.

Server-Sent Events (SSE) enables real-time communication between the server (PHP script) and the client (JavaScript) by processing and sending data in chunks as they arrive. This results in a live stream of data from the API that can be efficiently and seamlessly displayed or manipulated on the client-side.

5. Displaying Readable Text with Javascript

The PHP file works to parse the stream, but it’s still not in a human-readable format. To create a client-side interface, set up an HTML layout that includes a div with the id "streamingText". This is where the API response will appear as real-time typing.

<html>
<body>
<div id="streamingText" style="font-size: 22px; overflow: scroll;"></div>
</body>
</html>

Use JavaScript to send the user input through the URL and request text information. The following JavaScript snippet illustrates how to communicate with the server-side PHP file and handle the results:

<script>
// Set your request input
const yourInput = 'Generate example text of medium length, talking about the feature of API response streaming to resemble real time typing, from the ChatGPT API.';

// Send the input data as a query parameter to your PHP script
const url = `ajax.php?input=${encodeURIComponent(yourInput)}`;

// Create a new EventSource object for real-time server-sent events (SSE)
const source = new EventSource(url);

// Initialize the response content and current index for displaying typed characters
let responseContent = '';
const streamingTextElement = document.getElementById('streamingText');
let currentIndex = 0;

// Function that types out received chunks character by character
function typeCharacter() {
  if (currentIndex < responseContent.length) {
    streamingTextElement.textContent += responseContent.charAt(currentIndex);
    currentIndex++;
    setTimeout(typeCharacter, 100); // Adjust the typing speed by changing the delay (in milliseconds)
  }
}

// Process the received SSE events
source.onmessage = function (event) {
  const data = JSON.parse(event.data);

  if (data.id) {
    // Process the data message chunk
    const choices = data.choices;
    if (choices && choices.length > 0) {
      const content = choices[0].delta.content;
      if (content) {
        responseContent += content;
        typeCharacter();

        // Auto-scroll to the bottom as new content is added
        streamingTextElement.scrollTop = streamingTextElement.scrollHeight;
      }
    }
  } else if (data === '[DONE]') {
    // All data received, logging it and closing the connection
    console.log('All data received:', responseContent);
    source.close(); // Close the connection
  }
};

// Handling errors in receiving SSE events
source.onerror = function (error) {
  console.error('Error:', error);
  source.close(); // Close the connection
};
</script>

6. Implementing the Real-Time Typing Effect

To simulate real-time typing, use the setTimeout function in your JavaScript code. This function lets you define a delay between separate characters, creating a visual effect that resembles live typing.

This snippet is already included in the longer code above, but here it is again so you can see what’s going on:

// Function that types out received chunks character by character
function typeCharacter() {
  if (currentIndex < responseContent.length) {
    streamingTextElement.textContent += responseContent.charAt(currentIndex);
    currentIndex++;
    setTimeout(typeCharacter, 100); // Adjust the typing speed by changing the delay (in milliseconds)
  }
}

7. Handling Errors and Stream Completion

Proper error handling is essential for any application. The onerror event handler logs errors, while closing the connection in case of issues. An event message signaling completion helps close the connection when there is no more data to receive.

// Handling errors in receiving SSE events
source.onerror = function (error) {
  console.error('Error:', error);
  source.close(); // Close the connection
};

Conclusion

Real-time text streaming using OpenAI’s ChatGPT API, PHP, and JavaScript enables the smooth visualization of content, increases user engagement, and handles data efficiently.

The example provided in this blog post can be adapted for many use cases, including chatbots, live content generation, and other dynamic applications. Experience the transformation real-time streaming can bring to your web applications and explore countless possibilities for unique user experiences!