Category: Ollama

  • Creating an AI generated blurb in WordPress using Ollama

    So I have never been much of a writer, so when it came time to write my bio for the website I simply fed my resume into El3ktra (my local Ollama server) and asked her to write it for me. Then I thought, wouldn’t it be fun if the website generated a new bio each page load?

    The first step was to make a text-only version of my resume for my AI to consume while building my bio, which I stored in the root of my directory.

    The next step was adding php code that would talk to my Ollama server and generate a bio as a shortcode function. This can be done by editing the functions.php, found in /var/www/html/wp-content/themes/<your-theme>

    function get_bio() {
        // get the resume text
        $url = "cv.html";
        $resume_txt= file_get_contents($url);
    
        if ($resume_txt!== false) {
    
                $data = [
                    "model" => "llama3.1:8b",
                    "prompt" => "Use Kim's Resume to write a short professional bio in the third person for Kim, less then 200 words, to put on her website.  Here is Kim's resume: " . $resume_txt]
                    "stream" => false
                ];
    
                $ch = curl_init("http://localhost:11434/api/generate");
                curl_setopt($ch, CURLOPT_POST, true);
                curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data));
                curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
                curl_setopt($ch, CURLOPT_HTTPHEADER, ['Content-Type: application/json']);
    
                $response = curl_exec($ch);
                curl_close($ch);
                $r = json_decode($response)->response;
                if (str_contains($r, ':')) {
                        return substr(strstr($r, ':'), 2);
                } else {
                        return $r;
                }
    
        } else {
            return "Failed to retrieve the resume.";
        }
    }
    add_shortcode('bio', 'get_bio');

    After that, it was a matter of adding shortcode bio and viola, lovely bios generated every load.

    Gotchas

    The issue that I ran into was that I couldn’t get my AI to not add an introduction for the bio, ie “Here is a bio for Kim: Kim is a programmer etc..." no matter how much I begged her not to in the prompt.

    The following code fixes that in a dirty way by checking to see if there is a colon in the answer, and returning everything after that colon. Yes, dirty but effective.

    if (str_contains($r, ':')) {
                        return substr(strstr($r, ':'), 2);
                } else {
                        return $r;
                }

    Works, but too slow!

    I was excited to see generated bios, but these took several seconds to generate the text and meanwhile my page was just sitting there. My next step was to load the shortcode after the page was done loading. For this we needed JQuery and AJAX.

    The first step was to write the function that would call the shortcode (in the function.php file):

    function handle_delayed_shortcode() {
         echo do_shortcode('[bio]');
         wp_die(); // Always close AJAX connections in WordPress
    }

    In the same file, I registered AJAX actions for both logged-in and anon users:

    add_action('wp_ajax_load_bio', 'handle_delayed_shortcode');
    add_action('wp_ajax_nopriv_load_bio', 'handle_delayed_shortcode');

    Finally, I replaced the shortcode on my site with the following custom HTML:

    <pre class="wp-block-code"><code><div id="delayed-shortcode-container">Loading...</div>
    <script>
    jQuery(document).ready(function($) {
    
        $.ajax({
            url: 'https://el3ktra.net/wp-admin/admin-ajax.php', 
            type: 'POST',
            data: {
                action: 'load_bio'
            },
            success: function(response) {
                $('#delayed-shortcode-container').html(response);
            }
        });
    });
    </script></code></pre>

    Basically it waits for the page to finish loading, then calls the shortcode and writes the result to the delayed-shortcode-container div. Pretty simple.

    Gotchas

    Now one of the first things that I ran into, using the TwentyTwentyFive theme, is that JQuery is not loaded to this theme. I determined that JQuery wasn’t running by adding an alert to the JQuery code. I had to add this to the end of my functions.php file:

    add_action('wp_enqueue_scripts', function() {
    wp_enqueue_script('jquery');
    });

    This allowed JQuery to run on my page.

  • Managed to Secure my Ollama/Whisper Ubuntu Server

    So I am a novice web administrator running my own server, which hosts apache2, ollama, and whisper. I have programs that need to access these outside my local net, and I was as shocked as many are to find that there isn’t a built in way to authenticate Ollama,

    I was able to get this working using Caddy. I am running Ubuntu 24.04.1 LTS, x86_64. Thanks to coolaj86 (link to comment) who got me down the right path, although this solution didn’t work for me (as I am already running an apache2 server and didn’t want to use Caddy as my webserver.)

    First, I installed Caddy:

    curl https://webi.sh/caddy | sh

    Then I created a few API keys (I used a website) and got thier hashes using

    caddy hash-password

    Finally, I created Caddyfile (named exactly that):

    http://myserver.net:2800 {
    handle /* {
    basic_auth {
    email1@gmail.com <hash_1>
    email2@gmail.com <hash_2>
    email3@gmail.com <hash_3>
    }
    reverse_proxy :5000
    }
    }
    http://myserver.net:2900 {
    handle /* {
    basic_auth {
    email1@gmail.com <hash_1>
    email2@gmail.com <hash_2>
    email3@gmail.com <hash_3>
    }
    reverse_proxy :11434
    }
    }

    Started up Caddy:

    caddy run --config ./Caddyfile &

    And ports 2900 and 2800 were no longer accessible without a password. Ports 11343 and 5000 are closed both on my router and ufw and are not publically accessible at all. To access Ollama, I had to go through port 2900 and supply a username (my email) and the api key I generated.

    The next step was to update my code to authenticate, which I haven’t seen spelled out anywhere although it’s pretty obvious. I am using Python.

    Here is what my python Whisper request looks like:
    resp = requests.post(url, files=files, data=data, auth=(email, api))

    And here is what my python Ollama Client call looks like (using Ollama Python):

    self.client=ollama.Client(host=url, auth=(email, api))

    I hope this helps! the next step is obviously to send the requests via https, if anyone has thoughts I’d love to hear them.