Articles tagged 'development'

Page 1 of 4

    Setting up a static site with basic web authentication

    Moving on from my initial Heroku experiment, I removed the Jekyll site and now have the bare minimum required for hosting content requiring authentication on Heroku

    1. Create a new Heroku app
    2. Add the [static site buildpack]
    3. Create a folder (I called mine static) and create a file called static.json with the following contents { "clean_urls": true, "https_only": true, "root": "static/", "basic_auth": true }
    4. Set two environment variables BASIC_AUTH_USERNAME and BASIC_AUTH_PASSWORD - make sure that the value of password is readable by htpasswd e.g. openssl passwd -apr1 <password_plaintext>
    5. Copy whatever content you are going to host into the static folder, add this to git and push to heroku using git push heroku master (Heroku will have set up that remote when creating the app). When regenerating the site this is the only step you will need to rerun.

    I have a lot of Gitbook content which I will need to add to the site - for now I am manually building the books and copying them into the static folder. Next step will be to script that.

    posted on April 2, 2020herokudevelopment


    Setting up a static site with basic web authentication

    I have some Gitbooks created for notes that I've taken on various programming books or courses down throughout the years. I didn't want to put these up on a Github pages site as notes like that probably violate copyright. So I wanted some way to hide them behind authentication so that only I could see them.

    My initial thinking is that I would need some kind of server to respond to requests and only serve the content if needed. You could probably do this with S3 or some other AWS service but I'm paranoid about using those as even if you think you are within the free tier, you could still be hit with a bill for unexpected usage. I didn't trust that my site would always be configured correctly enough to avoid that. Then next I considered a site that would give me enough free CPU to run a basic server. Enter Heroku.

    I tried to use the Jekyll Auth plugin but couldn't get it working.

    • The gem released on rubygems is out of date and clashes with the dependencies in Jekyll 4.0. On the upside this did teach me how to include a gem directly from Github. If the repository in question has a .gemspec file at the root level then you can use the git option and pass the url of the repo e.g. gem 'jekyll-auth', git: "https://github.com/benbalter/jekyll-auth"
    • Even after getting everything configured I was getting denied when trying to authenticate with my Github account. It looked similar to this issue which didn't get resolved so I decided to try something else.

    Heroku Buildpack

    I tried a different approach which was to get Heroku to serve the files and then use http basic authentication to protect them. Heroku has a buildpack for static sites. This is marked as experimental but it worked fine for me. Hopefully they won't remove it.

    • I used the guide here to get a basic Jekyll installation up and running on Heroku. This worked pretty much as outlined.
    • Then I add http basic authentication as outlined in the relevant section on the Heroku build pack page. You add the option to the static.json file and specify the username and password that you want in env variables. I only wanted a single user i.e. me so I'm not sure if this approach scales to multiple users. I think it does via htpasswd files. To set the password you must first run it through a tool to generate a hash that htpasswd can accept. I used openssl passwd -apr1 <password_plaintext>. Then you set the env variable BASIC_AUTH_PASSWORD to be this value. This didn't work for me from the command line - probably some characters that needed escaping - so I just set it via the web interface.

    Some misc Heroku issues that I ran into.

    • I work on WSL and the snap approach for installation didn't work for me so I used the curl script. This mostly worked, however the permissions on a couple of folders were set to root so I had to chown these back to my normal account.
    • I got errors about an app that I thought I had deleted. However the Heroku CLI adds a remote to the git repo pointing to the app so even if you delete the app, make sure that the remote is deleted also.

    In the end this worked out fine - a lot easier than I expected and I have a website hidden behind authentication. The dyno that the website is on goes to sleep if not used for a while so the initial request for a page can be slow if the dyno is loading up but it's still only a few seconds. Now that I have a server on Heroku I look forward to seeing what else I can do with it.

    posted on March 18, 2020herokudevelopment


    Excellent Series of Articles on Modern C++

    Here is a great series of articles on modern C++:

    1. Relation between C and C++, Strings
    2. Namespaces, Classes, RAII, Smart pointers, Threads, Locking, Error Handling
    3. Inheritance & polymorphism, Templates
    4. Lambdas, Containers and algorithms, Boost containers
    5. Memory management, RVO, Smart Pointers
    6. Static assert, Constexpr, Time, Enums, Initializer Lists, Regexp, Optionals

    posted on April 5, 2019development


    Automated building with CircleCI

    I've started using CircleCI to automatically build repos - specifically my various static websites. It's working really well. CircleCI will detect a change to the Github repo, automatically trigger a build and then push the result to the gh-pages branch.

    One tip I found useful is to use a machine account to push to the gh-pages branch - this means that we don't need to use our actual github SSH key. You will need machine accounts in [both[(https://github.com/DevProgress/onboarding/wiki/Using-Circle-CI-with-Github-Pages-for-Continuous-Delivery) Github and CircleCI. For each repository that you want to build you need to add the machine account as a collaborator on the Github repository.

    CircleCI will also trigger a build on the push to the gh-pages branch which is not what you want. To stop this we need to add [ci skip] to our commit message. In Middleman we can do this by adding deploy.commit_message = "Automated commit at #{time} by #{signature} [ci skip]" to our config.ru.

    posted on October 2, 2018development


    Misc HTML Links

    Some links gathered while adding content to my chess site

    posted on October 24, 2017development


    Sublime Text Vintage Mode

    I started learning Vim recently and it works fantastically well on Linux. Windows is a bit of a mess though and I'm not confident of getting plugins working correctly. Maybe the new Linux subsystem for Windows will help this.

    In the meantime I've enabled vintage mode on Sublime Text and it seems to be working well. The main thing it gives me is the keyboard navigation keys. Not having to use the mouse or arrow keys is a big win. I never realised how good it would be until I tried it.

    References: Sublime Text Docs, King Luddite Blog Post

    posted on June 1, 2017development


    Converting Yaml to JSON in JavaScript

    I use the Middleman data files feature for my chess site. These files are written in yaml. I could have used json directly for these which would have removed the need for this post but I find yaml is easier for manual editing - with json you constantly have to worry about matching up brackets. From the Middleman erb files, when I build locally I am able to access things like data.fide.ratings. However I would like to be able to load this data remotely and process it from my javascript code.

    To load the remote file I just use the JQuery Ajax get function and pass it the url of the file that I want. Then I use the js-yaml library to parse the result and create a data json object from this. Now from my JavaScript code I am able to access the same data using the same notation.

    function loadYamlFromUrl() {
        $.get( "https://raw.githubusercontent.com/gerardcondon/chess/master/website/data/ratings.yaml", function( data ) {
            var data = jsyaml.load(data);
            // can now access data.fide etc
        });
    }
    

    This has worked well for me and for any data file that I was able to access via Middleman, I can now access the same data from JavaScript in the same format.

    posted on February 25, 2017development


    Adding sidebar links to local Gitbooks

    I've started keeping track of various notes in markdown format and I wanted to use Gitbook to group them together. I'm using the npm module to build it locally. It's a useful tool and theThe completed book looks nice. However I would like to add some customisations e.g. links back to my main page.

    It's not easy to add simple customisations. Or more like it probably is but it's not well documented how to do so. Editing the header is a bit awkward as you have to modify the theme yourself. I'll probably end up just adding a header above it using an iframe.

    One thing I did find out how to do was to add links to the sidebar. You edit the book.json to add a links section like this

      {
        "links": {
          "sidebar": {
            "Gerard Condon": "http://www.gerardcondon.com",
            "Github": "https://github.com/gerardcondon/tools-docs"
          }
        }
      }
    

    posted on February 15, 2017development


    Adding microposts to Middleman

    I came across the idea of microblogs from Manton Reece's blog. The idea is that instead of just using the blog for long content, you can post short Tweet like items there also. Needless to say you can just use Twitter for this but it's more fun to do something unneccessarily complicated! I have form here - like using Octopress for a blog instead of WordPress.

    Basically, instead of using Twitter to publish short notifications, you can post them to your own site and if necessary to the likes of Twitter from there. Having control of your own content gives you independence from third party sites - if Twitter shut down tomorrow then my posts would still be available here. If I'm putting content on the web then ideally I would like it to be on my site under my own domain name. I've decided to add them first to my chess site and then if they work out there, try to port them to Octopress.

    That blog is written using Middleman, which has the ability to use data files. Any yaml file in the data folder is available to the Middleman app using a data.filename.foo like syntax. This means that I can create a yaml file containing the microblog posts and then use that to generate the appropriate html pages.

    I have a microposts.yml file with each entry having text and date entries. This is sufficient for the moment but also yaml is flexible enough that I can add more attributes later. yml - text: here is another post date: 2016-11-19 - text: here is a post date: 2016-11-20

    I use the Middleman blog extension to generate the blog portion of the site. I want to keep using this as it handles rss, tags, archive pages, pagination etc. It works by generating posts from files in a source folder. In order to integrate with this, I want to generate a file per micropost in the correct location. Under the source folder I split out my posts into two folders manual and automated. I updated the regex in config.rb to handle this blog.sources = "posts/{type}/{year}-{month}-{day}-{title}.html". Middleman will process these equally but now I can check what type the post is by using data["type"].

    In the config.rb I delete the contents of the automated folder on each build. I tried using before_build hooks but they were being called too late - the Middleman data structures were already created with the previous contents of the automated folder. This lead to build errors and issues with sitemaps etc. So I just perform the File operations directly in the config.rb. This ensurs that they are executed before the blog extension processes the posts folder. Then I iterate through the micropost data from the yaml file and create the appropriate file in the automated folder.

    # config.rb - Clear out automated posts folder
    automated_posts_folder = "#{config[:source]}/posts/automated"
    FileUtils.rmtree(automated_posts_folder)
    FileUtils.mkdir(automated_posts_folder)
    
    post_counter = 1
    data.microposts.each do |micropost|
      post_content = "---\n"
      post_content += "title: mp#{post_counter}\n"
      post_content += "date: #{micropost.date}\n"
      post_content += "tags: microposts\n"
      post_content += "micropost: true\n"
      post_content += "---\n"
      post_content += "#{micropost.text}"
    
      File.write("#{automated_posts_folder}/#{micropost.date}-mp#{post_counter}.html.md", post_content)
      post_counter += 1
    end
    

    I've added micropost: true metadata to each of my generated files so that I can check if the post is a micropost using if data["micropost"]. This allows me to customize how I display these posts - in particular I don't show a title for these posts.

    Speaking of titles, some special handling is required for these as the Middleman blog extension uses these for the generated html file names. I added an attribute display_title to the BlogArticle class which will return an empty string if the post is a micropost. This allows me to specify a fake title for each micropost using an incrementing ID counter. The blog extension can use this fake title as normal but it will never be seen by the viewer of the blog. Instead anywhere that a title is required in html markup I can change it to use the display_title attribute. The fact that Ruby has open classes makes adding this very easy - I can do it in the config.rb file and don't need to go mucking about in the Blog extension code.

    # config.rb
    module Middleman
      module Blog
        module BlogArticle
          def display_title
            data["micropost"] ? "" : data["title"]
          end
        end
      end
    end
    

    This means that I can display these microposts just using the main text and not displaying a title. They interleave nicely with longer form blog posts.

    Using the Middleman Blog Extension means that I also get RSS integration for these posts. I edited the xml templates to remove the title if the post is a micropost. Now in my RSS reader, they show up with the body text as the title instead of the fake title needed to make the extension work.

    This current implementation is basic enough and I have a few ideas of how to improve it in future.

    • I like the idea of the static site but the editing part is not particularly nice and is a bit of a barrier to posting. I would like to use something like Contentful so that I can add posts without having to open a dev environment.
    • Once you have something like Contentful, then the next step would be to use that to trigger automatic builds once a new post is added. This would mean that the site is always updated and assuming Contentful works well from Android, would allow me to make these microposts from my phone.
    • I could use IFTTT to cross post the microposts to Twitter - however I would have to deal with the 140 character limit there. I would also like to automatically post a notification to Twitter when I put up a normal blog post.
    • Currently everything is going into the same RSS feed. I would like to add separate RSS feeds e.g. micropost only, normal post only. I think this would also help with using IFTTT to post to Twitter.
    • One nice consequence of using the Middleman data files is that there is a much better separation of content from markup. I would like to expand this to also apply to full blog posts. Ideally everything content related would come from the data files. Then I have more options as to how to generate those data files e.g. Contentful for the tweets or a Rails app for the pgn files.

    posted on November 20, 2016development


    Fixing Twitter sidebar for Octopress 2.x

    I noticed recently that the Octopress Twitter plugin for my sidebar was broken. The reason is that Twitter no longer supports the APIs that the plugin accesses. Instead now they have a widget page to generate html code that will embed a list of your tweets on your page.

    I created a widget for myself and customised it by adding the nofooter transparent noheader attributes to data-chrome. These remove various headings and background colouring so that it blends in with the site a bit more.

    To add this to an Octopress site, you replace the code in source/_includes/asides/twitter.html with this new code. I also added a follow button which displays a Twitter formatted follow button and a count of your subscribers.

    The full code is as follows

    <!-- source/_includes/asides/twitter.html -->
    {% if site.twitter_user %}
    <section>
      <h1>Latest Tweets</h1>
      <a class="twitter-timeline" data-chrome="nofooter transparent noheader" data-tweet-limit="5" href="https://twitter.com/gercondon">Tweets by gercondon</a> <script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
      <a class="twitter-follow-button" href="https://twitter.com/gercondon">Follow @gercondon</a>
    </section> 
    {% endif %}
    

    posted on November 9, 2016developmentoctopress


    New Chess Site

    I wanted to learn how to create websites using Bootstrap so I've created a chess site at www.gerardcondon.com/chess. This is a static site developed using Middleman. It uses Bootstrap and pgn4web to display chess games and allow the viewer to play through them.

    posted on December 9, 2015developmentchess


    Upgrading to Rails 4.2 on OpenShift

    I updated my Rails test app from 4.1 to 4.2. However when I pushed to OpenShift I got the following error

    You have already activated rack 1.5.2, but your Gemfile requires rack 1.6.0. 
    Using bundle exec may solve this. (Gem::LoadError)
    

    I found the answer on Google Groups here. The root cause is that OpenShift is dependent on Rack 1.5.2 and Passenger 4.0.18. The proper fix will have to wait until they update their versions of that software. Until then to fix this error, ssh into the OpenShift app and in the app-root folder run

    gem install rack
    

    posted on June 15, 2015development


Next page

Tags

By year

  1. 2020 (14)
  2. 2019 (17)
  3. 2018 (2)
  4. 2017 (11)
  5. 2016 (3)
  6. 2015 (6)
  7. 2014 (3)
  8. 2013 (11)
  9. 2012 (25)