Billal BEGUERADJ

The broken window theory

Sat, 14 Feb 2026 12:33:21 +0800

Two cars without licence plates were parked in 2 different towns. Both cars were intact except for one of them which was left with a broken window.

Within a dozen minutes, a family came to the car with a broken window to remove its battery and radiator. After 24 hours, nothing of value was left in that car. Fast forward, random acts of destruction followed.

Meanwhile, in the other town, the fully intact car remained as it was left for a week; until the owner decided to break one of its windows. The car then quickly endured the same fate as the previous one.

This was an experiment conducted in 1969 by Philip Zimbardo, an American psychologist.

This is a phenomenon we are familiar with in software development: once we tolerate a bad design decision, poor code and such, the software tends to rot by time. This is similar to when thermodynamicists observe the steady entropy increase in the universe. Entropy is the amount of disorder in a given system.

The way we overcome this issue in software development is to address and fix the issue as soon as it is diagnosed.

The broken window theory remains valid not only when it comes to software development but practically in every aspect of our lives:

An urban area where littering, graffiti and drug sale are not fought at the initial stage becomes a crime nest.
If you start your morning leaking your focus, thoughts, emotions or energy to noise, you will just tread water for the rest of the day. You won’t sleep well either.
Tolerating one imperfection in your craft will encourage you to introduce more defect, affecting its quality.
Not facing disrespect -as if there is a virtue in being a rabbit- attracts more of it.

The only counter-example which invalidates the broken window theory is a relationship from a man’s perspective: if problems start to appear in your relationship don’t try to fix them. You won’t thrive on that. A woman is born to discover herself. A man is born to invent himself.

Python decorators

Sun, 13 Jul 2025 12:33:21 +0800

What are the origins and goals of Python decorators? How to implement them and why are they useful?

Introduction

Even when dealing with scientific and technical subjects, your subjectivity rules over you. I mean, each person has her own style in learning and explaining ideas. This subjective aspect is shaped not only by your intellectual capacities but more importantly by your personality. By personality, I mean how you see and interpret things. This subjectivity dimension highly impacts your learning curve. This is what you may already have observed in yourself. This is also what the Swiss development psychologist Jean Piaget detailed in his theory of cognitive development. So maybe the best book dealing with Python decorators would not fit you. Thus, you understand the reason behind this an other, yet different, contribution on Python decorators that hopefully will fit you.

The first thing I would like to start with is that Python decorators have nothing to do with the decorator design pattern described by the Gand of Four in their famous and now classic book Design Patterns: Elements of Reusable Object-Oriented Software.

Back to 2005

If you are like me when I have to learn new technical notions, you may ask yourself about the origin and goals of Python decorators. As you may already have read elsewhere, Python decorators were first implemented in Python2.4 when introducing @classmethod and @staticmethod built-ins.

I read an article published on 2005 by the author of Python Web Server Gateway Interface specification (PEP 333) in which he said: as a relatively new feature, their full possibilities have not yet been explored, and perhaps the most exciting uses haven’t even been invented yet.

As you can see, it is not always evident to see the importance of a new concept, especially if you are not the one who conceived it. Philip Edy had hard time to find the necessity to use decorators. To highlight the 2 reasons as why you need to implement decorators, he invoked a short code snippet without a decorator:

import atexit

def goodbye():
   print "Goodbye, world!"

atexit.register(goodbye)

and then with a decorator version:

import atexit

@atexit.register
def goodbye():
   print "Goodbye, world!"

While I agree with the first advantage, I do not agree with the last one in that repeating the function name in this situation does not go against the DRY principle you have learned in The Pragmatic Programmer: From Journeyman to Master. The DRY principle is rather violated in this highly rated answer I randomly picked from StackOverflow:

class Date(object):

    day = 0
    month = 0
    year = 0

    def __init__(self, day=0, month=0, year=0):
        self.day = day
        self.month = month
        self.year = year

This class is wrongly designed: You can see that the information hold by both the class and instance variables are of the same nature. The class variables must be removed because each day is unique. Not only this does not conform to DRY principle but it is also a confusing design.

If I do not agree with Phillip Eby then what makes decorators a unique and useful Python feature? I will answer this question by the end of this article because you can not grasp the meaning of what I say without actually introducing what decorators are.

Definition

Unfortunately, I see many resources making a parallel between decorators and Java annotations. I say unfortunately because the only common aspect between these 2 notions is the syntactic sugar line itself which looks exactly alike. Everything else is different:

Java annotations have no effect on the method or class they decorate, unlike Python decorators which are functions able to modify the behavior (but not the definition) of other functions arbitrarily at runtime.
Java annotations provide metadata to class or method while Python decorators take a function as an argument and return a function.

I often read comments on other online Python communities complaining about the fact decorators are simple to use but difficult to implement. The other thing I hear is about the simplicity of implementing decorators if you develop in Python under the OOP perspective. So let us take some useful steps that will lead and help to understand what Python decorators are about.

Python functions

Save the following simple Python function in a file called hello.py:

def greet():
   print('Hello')
if __name__ == '__main__':
   greet()

Running python hello.py will generate a bytecode which Python runtime loads to get interpreted by the virtual machine which then creates a function object to which the name greet is binded.

FCO

I am not here to teach you what is a function. I suppose you already know that. But it is important to remember that in Python, all functions are CFO. But what is this weird term? Well, actually, many programming languages implement the notion of first-class functions (CFC). This means that in such programming languages you can, among other things, use functions as parameters as well as return values which you can store in a variables. This useful concept was introduced by Christopher Strachey. But because everything in Python is an object (yes, including classes!), we rather need to use the acronym FCO instead of FCF. FCO sands for first-class objects.

If you do not believe that functions are objects in Python, add this line to the previous hello.py code:

`1`	`print(greet)`

This will result in printing:

`1`	`<function greet at 0x7fd2d1fb32f0>`

This output simply means that the virtual machine has allocated a memory address 0x7fd2d1fb32f0 for the object called greet.

A function can be passed as an argument to a function:

def greet(function_name):
   """This functions takes as an argument an other function.
   Only the function name is passed
   """
   # Call to function_name and printing out the value it returns
   print('Hello {} !'.format(function_name()))

def name(fname = 'Jack'):
   """This function takes a named and optional argument with a default value.
   It returns the passed argument.
   Well, useless example, but this is just for illustration.
   Once you understand the basics, you can code your own magical stuff.
   """
   return fname
if __name__ =='__main__':
   greet(name)

The above example does not only show that we can pass a function as an argument to an other function:

`1`	`greet(name)`

but it also shows that you can use a function as a return value to store in a variable:

`1`	`print('Hello {} !'.format(function_name()))`

In case you are stubborn and still do not believe that functions in Python are first class objects then add this line to the previous program:

1
2

print(greet.__class__)
print(issubclass(greet.__class__, object))

The last line above will print True because the function greet, as all other objects in Python, inherits from the object baseclass.

Nested functions

Python allows you to define -and call- a function inside an other one. This feature is what is commonly called nested functions. So let us see nested functions in actions by saving the following program in a email.py file:

import re

def email(email_address):
   """This is an outer function.
   It receives an arbitrary email address.
   It delegates the validation check to is_valid() inner function
   """
   def is_valid(email_address):
       """This is an inner function.
       It has access to the variable of the enclosing scope
       to check its validation.       
       """
       ok = re.search(r'(^[a-zA-Z0-9\.\+_-]+)@[a-zA-Z].[a-zA-Z\.]+$', email_address)
       return True if ok else False
   #This will prints True or False
   print(is_valid(email_address))

# Main program
if __name__ == '__main__':
   email('begueradj@begueradj.com')

Note that you can nest your functions as deeply as you want as long as you respect variables scope.
Dealing with nested functions, it is important to keep in mind the variables scope. If you need to modify the value of a variable enclosed within the namespace of an outer function and lays inside its inner function, then you must declare it with nonlocal keyword, otherwise the Python interpreter will create a variable local to the inner function:

def outer():
   x = 1
   def inner():
       nonlocal x #This means you want to access the x defined in outer()
       x = 2
   inner()
   print('x = {}'.format(x)) #The value of x is no longer equal to 1
   
# Main program
if __name__ == '__main__':
   outer()

If you got a syntax error after running the above program, this means you used Python2.x while the keyword nonlocal is introduced in Python3.x. Do not confuse nonlocal with the global keyword which rather is used to reference variables define in the scope of a module.

Closures

Let us turn back to our previous email.py to tweak it a little bit like follows:

>>> import re
>>> def email(email_address):
...    def is_valid(): #Changed only this
...        ok = re.search(r'(^[a-zA-Z0-9\.\+_-]+)@[a-zA-Z].[a-zA-Z\.]+$', email_address)
...        return True if ok else False
...    return is_valid
... 
>>> result = email('begueradj@begueradj.com')
>>> result()
True
>>> del email
>>> result()
True
>>>

Do not you notice something weird? Well, actually there are two unexpected behaviors:

The result of first call to result() is not expected because email_address is local to email(). This means any result of an operation made on email_address could no longer exist after email() is executed. But you clearly see this is not the case because is_valid() is called only after the return of email().
After deleting email() we still got information about the result of the operation done on its local variable email_address! Local variables are supposed to disappear after the execution of their enclosing scope! This is weird enough!

Actually this is achieved thanks to a feature of Python called closures. A closure consists of a nested function must point to a variable which is local to the enclosing function which must return the nest function. I introduced closures here because they are extensively used for Python decorators.

Maybe you can understand the Python shell better:

1
2
3

>>> result.__closure__
(<cell at 0x7f80c224d7f8: str object at 0x7f80c04af228>,)
>>>

Decorators

If you have understood the previous secions, you are already equiped to learn properly about the decorators:

Definition

I see lot of resources misleading newbies by telling them that the decorator is that weird notation in the form of. Other resources give a more formal definition that may not be that easy for a beginner to understand. Albert Einstein once wrote (or said): If you can’t explain it to a six year old, you don’t understand it yourself. So let me define what is a decorator in plain English: a decorator is a function which takes an other function as an argument in order to modify its behavior. Yes, as simple as that.

There are two formal definitions of decorators that maybe you want to learn about anyway. If this is your case then you have to know the first and almost instinctive formal definition commonly used to describe decorators is: a decorator is a callable returning an other callable. Do not be scared of the callable notion here. In fact, as in any other programming language you already know, you can call a function in Python; hence the adjective callable. More technically speaking, every object which implements the __call__() method is said to be a callable object.

The second formal definition you can hear about elsewhere is: decorators are a metaprogramming concept just like metaclasses. Honestly, this is not the best definition to provide for a newbie. But I let you know that metaprogramming simply means that one (or more) function or class of your program tries to modify other pieces (function, class) of your code. For the metaclass notion, I am not going to describe it here, but I may write a simple tutorial about it if you ask me through a comment below.

Example

As with other things in your daily life, the best definition you can provide is done by giving an example:

def decorator(decorated):
   """This decorator wraps a nested function to add
   more functionality to the decorated() function.
   """
   
   def inner():
       """Here is modified the behavior of decorated()
       This is achieved by adding functionality to it.
       """
       print('Hello and welcome to my page.')
       decorated()
       print('I am of Algerian nationality living in Paris and born in a commune situated in Bejaia called Feraoun')
       
   return inner   
    
def decorated():
   print('I am Billal BEGUERADJ.')
   
   
# Main program goes here
if __name__ == '__main__':
   about = decorator(decorated)
   about()

Yes, we have programmed our first decorator. The previous example respects the definition and principles of a decorator: the callable (function) decorator() takes an other callable I named decorated(), it uses a nested function which name is inner() and returns a callable (this later one). Does this remember you of closures? That is why I said previously that closures are extensively used to implement decorators.

The function decorated() is supposed to execute only this instruction: print('I am Billal BEGUERADJ.') but thanks to our decorator, we have decorated it to display a detailed self-representation.

def decorated():
   print('I am Billal BEGUERADJ.')
   
   
# Main program goes here
if __name__ == '__main__':
   about = decorator(decorated)
   about()

are equivalent to:

@decorator   
def decorated():
   print('I am Billal BEGUERADJ.')
   
   
# Main program goes here
if __name__ == '__main__':
   decorated()

I see many online tutorials teach you that @decorator is the decorator. That is very wrong. That line of code is what you may call decoration notation but it is not the decorator itself. I think the first advantage of decorators may be quite obvious: at least you reduced the size of your code (number of instructions) which thing is simply beneficial.

Decorators of functions with arguments

The previous decorator is quite simple because the function it decorates takes no arguments. So how to decorate a function with parameters?
Here is an example:

import math

def find_root(func):
   """This decorator adds functionality to get_root().
   """
   
   def inner(a):
       """The most important thing you have to do here is to pass
       always the parameters of the callable object to your 
       nested function.
       """
       print('The root of {}: '.format(a))
       try:           
           return func(a)
       except ValueError:
           print('Opps ! Can not find the root of a negative value !')
           pass
		   
   return inner

# Decoration notation goes here
@find_root
def get_root(a):
   return math.sqrt(a)
   

# Main program starts here
if __name__ == '__main__':
   get_root(-4)

As you may have noticed, the trick is easy to grasp: always insert the name of the function to be decorated without parameters or even parentheses. Then pass the parameters of the function to decorated to the decorator’s nested function. That is it.

Of course, you can incorporate all what I did in inner() into get_root(). For this specific example, you may think it is a matter of opinion. But when you will read below about the utility of decorators, you will change your mind.

Note that in case the function to decorate has an arbitrary number of parameters packed within a list or dictionary, you can use * and ** notations to respectively unpack them:

def decorator(decorated):
    def nested(*args, **kwargs):
        print("+ ----------------------- +")
        return decorated(*args, **kwargs)
		print("+ ----------------------- +")
    return nested

Chaining decorators

You can chain decorators. This means you can use multiple decorators to decorate the same function as you previously learned. To use such multiple decorated function, you can use this notation:

@decorator1
@decorator2
@decorator3
decorated_function(*args)

Decorators and OOP

Until now, you have seen, and hopefully, decorators in action. But you are likely to be asked by your employee to develop use object-oriented programming at your workplace. So how to implement decorators when you design the solution to your problem using classes?

Let me re-write the very first decorator example you learned previously under the umbrella of OOP:

class Decorator(object):
   def __init__(self, decorated):
       self.decorated = decorated
   def __call__(self):
       print('Hello and welcome to my page.')
       self.decorated()
       print('I am of Algerian nationality living in Paris and born in a commune situated in Bejaia called Feraoun')

@Decorator
def decorated():
   print('I am Billal BEGUERADJ')

if __name__ == '__main__':
   decorated()

To make the instances of a class callable, you need to use __call__(). To simplify things for you, think of __call__() as the equivalent of the nested function you learned from above examples.
Most newbie developers say decorators are easier to implement using classes.

What are decorators useful for?

Even if the example described above are very simply, there is one main area where you can not get ready of decorators: when you develop a framework. Everything else you read in books is primarily-based opinion.

When you develop a framework, it is useful to hide the inner implementations of the different functionalities you offer so that the user of your framework can focus in designing the solution to his actual problem. If you read about the architecture of Flask framework, you are already satisfied and surprised to be able to deal with client request so easily:

from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello, World!'

But you can implement decorators for a large range of needs such as when you want to measure the performance of two versions of your code by calculating the time its takes to give me an output:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# We need this module for time operations
import time

def duration(func):
   """You can use this decorator to calculate the time
   a function takes to finish its execution.   
   """
   def nested(*args, **kwargs):
       """This is the nested function in which
       the operations are, properly speaking, done.
       """
       # Save the starting time of the function
       start = time.time()
       # Execute the function
       func(*args, **kwargs)
       # Caclulate the difference between present time and the previous one.
       since = time.time() - start
       # Easy to read output goes here
       print('+ --------------------- +')
       print('{}() executed in {} ms'.format(func.__name__, int(since * 1000)))
       print('+ --------------------- +')
   return nested
   
@duration
def greet(a):   
   print('I just print my argument: {}'.format(a))

# Main program starts here
if __name__ == '__main__':
   greet(2)

Conclusion

Decorators are a flexible means to reuse code (compared to functions) and are more useful to interface functionalities for the users of your framework. You can google PythonDecoratorLibrary and you will land on a page providing many examples of decorators that you may reuse in your programs. Your comments are welcome.

Effective file streaming using AdonisJs

Sat, 29 Jun 2024 12:33:21 +0800

Preamble

The majority of upload libraries/frameworks process files multiple times when streaming to an external service such as Amazon S3. Their upload workflows are usually designed like so:

Process request files then save them to the tmp directory.
Move each file from the tmp directory to the destination directory.
Use the external service’s SDK to finally stream the file to the external service.

This process wastes server resources reading/writing single files multiple times.

That is what you can read on the official AdonisJs documentation. Here we learn how to implement that by streaming the files directly to the MySQL database.

Our stack

AdonisJs: to build the RESTful API server
Nuxt.js + Vuetify: to design a user interface to interact with our API.

Our app

And this is the simple user interface I want to create:

The left button will be used to upload a file to the RESTful API server. The right button will be used to download it and display it on the right side

Project setup

In my demo, the client and server code are set apart. My client application is handled by Nuxt.js:

`1`	`yarn create nuxt-app client`

and the server by AdonisJs:

adonis new server --api-only

On my Github project, I use Vuetify.js and Eslint.

Nuxt.js client code

Here is where the main code is located:

components/
├── PhotoUpload.vue
pages/
└── index.vue
layouts/
└── default.vue

In pages/index.vue, we will have this code:

<template>
  <photo-upload />
</template>

<script>
import PhotoUpload from '@/components/PhotoUpload.vue'

export default {
  name: 'MainPage',
  components: {
    PhotoUpload
  }
}
</script>

The essential part of the code is what the PhotoUpload.vue component contains. So I show here the full code and then will explain the main points:

<template>
  <v-container
    grid-list-md
    text-xs-center
    fill-height
  >
    <v-layout
      row
      wrap
      align-center
    >
      <v-flex
        v-if="url"
        xs12
        sm4
      >
        <span
          class="font-weight-bold"
        >
          Client
        </span>
        <v-img
          :src="url"
          contain
        />
      </v-flex>
      <v-flex
        xs12
        sm4
      >
        <v-text-field
          v-model="photoName"
          name="photo"
          outline
          background-color="blue"
          color="blue"
          label="Select image"
          append-icon="attach_file"
          @click="selectImage"
        />
        <input
          ref="image"
          class="hide-input"
          type="file"
          accept="image/*"
          @change="imageSelected"
        >
        <v-btn
          round
          color="indigo"
          @click="uploadPhoto"
        >
          <v-icon
            color="white"
          >
            cloud_upload
          </v-icon>
        </v-btn>
        <v-btn
          v-if="url"
          color="indigo"
          round
          @click="loadPhotoFromServer"
        >
          <v-icon
            color="white"
            class="download"
          >
            cloud_upload
          </v-icon>
        </v-btn>
      </v-flex>
      <v-flex
        v-if="url"
        xs12
        sm4
      >
        <span
          v-if="photoId"
          class="font-weight-bold"
        >
          Server
        </span>
        <v-img
          v-if="photoId"
          :src="`${$axios.defaults.baseURL}/photos/${photoId}`"
          contain
        />
      </v-flex>
    </v-layout>
  </v-container>
</template>

<script>
export default {
  name: 'PhotoUpload',
  data: () => ({
    url: '',
    photo: '',
    photoName: '',
    photoId: 0
  }),
  methods: {

    selectImage() {
      this.photo = this.$refs.image.click()
    },

    imageSelected(e) {
      this.$emit('input', e.target.files[0])
      this.photo = this.$refs.image.files[0]
      this.photoName = this.photo.name
      this.url = URL.createObjectURL(this.photo)
    },

    async uploadPhoto() {
      const data = new FormData()
      data.append('file', this.photo)
      const config = {
        headers: {
          'content-type': 'multipart/form-data'
        }
      }
      await this.$axios.$post('photos', data, config)
    },

    loadPhotoFromServer() {
      this.photoId = 1
    }
  }
}
</script>

 <style scoped>
 .hide-input {
     display: none;
 }
 *{
     text-transform: none !important;
 }
 .download {
     transform: rotate(180deg)
 }
 </style>

Let us break down this mess. Most the above code was explained in the previous article. We added two small aspects to it though. First, we display the image we choose to send to the server. This image preview functionality in Nuxt.js can be useful in many cases you are enough smart to guess. For that, we draw some place holders in the form of Vuetify components to show it up when it is ready:

<v-flex
  v-if="url"
  xs12
  sm4
>
  <span
    class="font-weight-bold"
   >
     Client
  </span>
  <v-img
    :src="url"
    contain
  />
</v-flex>

As we can see, the image will be displayed only if url is loaded v-if="url". This famous url is just a variable we declared in the data() method, and we set its content to point to the local photo selected by the user using: URL.createObjectURL(this.photo).

We also added a Vuetify component to display the photo sent back by the server on a button click:

<v-flex
  v-if="photoId"
  xs12
  sm4
>
  <span
    class="font-weight-bold"
   >
     Server
  </span>
  <v-img
    :src="${$axios.defaults.baseURL}/photos/${photoId}"
    contain
  />
</v-flex>

We set photoId to 1 in loadPhotoFromServer() because we are interested to display the file which ID equals to 1 in the MySQL database. We use $axios.defaults.baseURL to refer to the baseURL key we defined in nuxt.config.js file to point to the URL on which our AdonisJs API is being served:

<v-btn
  v-if="url"
  color="indigo"
  round
  @click="loadPhotoFromServer"
>
  <v-icon
    color="white"
    class="download"
  >
    cloud_upload
  </v-icon>
</v-btn>

AdonisJs REST API server code

First of all, we need to create the migration, model and controller files for the photos:

1
2
3

adonis make:migration photos --action create
adonis make:model Photo
adonis make:controller Photo --type http

For our demo, let us be YAGNI and define only what we need in the migration file:

'use strict'

/** @type {import('@adonisjs/lucid/src/Schema')} */
const Schema = use('Schema')

class PhotosSchema extends Schema {
  up () {
    this.create('photos', (table) => {
      table.increments()
      table.specificType('file', "longblob").notNullable()
      table.string('type', 10).notNullable()
    })
  }

  down () {
    this.drop('photos')
  }
}

module.exports = PhotosSchema

Note that Knex.js which is used by AdonisJs here, does not have a predefined data field for images apart from binary. Luckily this same Knex.js offers us the possibility to define a data type of our choice as long as the RDBMS we use supports it using knex.specifiType() method to define the longblob table column.

Since are not interested in using the timestamps fields for our photos table, we should inform the corresponding model Photo.js about it:

'use strict'

/** @type {typeof import('@adonisjs/lucid/src/Lucid/Model')} */
const Model = use('Model')

class Photo extends Model {
  
  static get createdAtColumn() {
    return null
  }

  static get updatedAtColumn() {
    return null
  }
  
}

module.exports = Photo

The important part now comes from the PhotoController.js controller:

'use strict'

const Photo = use('App/Models/Photo')
// const Database = use('Database')
const getStream = use('get-stream')

class PhotoController {

  async store({ request, response }) {
    let photo = {}
    request.multipart.file('file', {}, async function(file) {
      const fileContent = await getStream.buffer(file.stream)
      photo.filecontents = fileContent
      photo.type = `${file.type}/${file.subtype}`
    })
    
    try {
      await request.multipart.process()
    } catch(e) {
      console.log(e.message)
    }
    
    const photoInstance = new Photo()
    photoInstance.file = photo.filecontents
    photoInstance.type = photo.type
    try {
      await photoInstance.save()
    } catch(e) {
      console.log(e.message)
    }
    response.status(201)
  }

  async show({ request, response, params }) {
    try {
      // const photo = await Database
      //  .table('photos')
      //  .select('file')
      //  .where('id', params.id)
      const photo = await Photo.find(params.id)
      response.header('content-type', `${photo.type}/${photo.subtype}`)
      response.header('content-length', Buffer.byteLength(photo.file))
      // response.send(photo[0].file)
      response.send(photo.file)
    } catch(e) {
      console.log(e.message)
    }
  }
  
}

module.exports = PhotoController

We used the store() method in the previous article. Here we only add the show() method which receives the photoId we mentioned previously as a paramter in order to use to fetch for the photo in the MySQL database. Of course, as for uploading the photo, we need to define the headers properly to let the client know which type of data it is going to handle.

We can use either the Database or the Query Builder providers to look for the photo in the database. Our code comments out the previous option so that we can use each one we prefer.

If you upload a photo, you will find it on the server, precisely in the folder /tmp/photos. Note that you have to enable CORS for the requests to be acceped. You can do that by setting cors: true in config/cors.js file. For larger images, set the limit value of maxSize to whatever you want in /config/bodyParser.js which is 20mb by default.

Note that we forgot to mention one important thing. Actually it is the first thing to set in the API because without it/them we can not communicate with the server: the routes. So in start/routes.js, we can define the routes to our photos resources as follows:

`1`	`Route.resource('photos', 'PhotoController')`

That is it. I made a GitHub repository for this demo if anyone wants to try it or make a pull request to improve it.

Python singleton design pattern

Tue, 25 Jul 2023 12:33:21 +0800

This article describes the main strategies to implement the Singleton design pattern in Python. I am going to skip the whole very subjective “use and do not use singleton” conundrum.

Introduction
Definition
Implementation
- Module
- Traditional
- Decoractor
- Metaclass
- Borg
Conclusion

Introduction

There are developers who simply call to “ban” this design pattern. Personally, I think while it does not respect the single responsibility principle, the singleton may still be useful in situations where you need to control a concurrent access to a shared resource, or want to use it to implement other design patterns.

Definition

On page 144 of the Gof4 book, the singleton design pattern’s stated intent is to “ensure a class only has one instance”.

Implementation

The singleton design pattern must be implemented in such a way that the sole class instance must be reachable via a global access point -namely a function, and can be extensible by the means of inheritence. Below are listed some methodologies Python uses to concrete it.

Module

The easiest approach to fulfill the singteton design pattern goal is to keep the global state in private variables within module which, in turn, must provide a way to access them by means of public functions.

As an example, here is singleton.py file:

_firstname = 'Begueradj' #private state

def get_firstname(): #access point
   return _firstname

The main application makes use of it as follows:

import singleton


class UseSingletonModule:

   def __init__(self):
       self.firstname  = singleton.get_firstname()
       self.lastname = 'Amazigh'
       
   def print_fullname(self):
       print('Full name: {} {}'.format(self.firstname, self.lastname))


if __name__ == '__main__':
   singleton_instance = UseSingletonModule()
   singleton_instance.print_fullname()

This is a practical approach but the main trivial issue with it is that there is no way to re-use the code through inheritance. This means the module approach does not fully respect the singleton design pattern. One can also argue this may sound a “coward” implementation in that may be most of other high level programming languages could do the same. This can also anger purists in that it is neither pythonic nor does it take advantage of the object-oriented concepts.

Traditional

To remedy to the drawbacks of the previous approach, there is a traditional way to implement the singleton design pattern in Python:

class TraditionalSingleton(object): 

   def __new__(cls): 
       if not hasattr(cls, 'instance'): 
           cls.instance = super(Singleton, cls).__new__(cls) 
       return cls.instance
   
   def __init__(self):
       self.data_test = 'begueradj'

Here an advantage is taken from the special static method __new__() which helps to control instance creation. Unlike __init__(), it always takes as a parameter the class cls in which it is invoked and returns an instance of it.

The Python built-in function hasattr() checks if the class cls has an attribute called ‘instance’. If that is not the case, the ‘instance’ attribute is created and returned.

A better explanation of this implementation can be through the use of TraditionalSingleton() class:

1
2

>>> first_instance= TraditionalSingleton() 
>>> second_instance = TraditionalSingleton()

Two instances of TraditionalSingleton() class are created. In reality, they are the same instance:

`1`	`>>> first_instance is second_instance True`

Hence modifying data_test variable in any instance will be automatically mirrored in the other instance:

1
2
3

>>> first_instance.data_test = 'begueradj.com'
>>> second_instance.data_test
'begueradj.com'

The main issue of this approach remains its inconvenience to use inheritance.

Decoractor

There are many ways to implement the singleton using a decorator. Here is one of the implementations I found on StackOverflow:

class Singleton:

   def __init__(self, cls):
        self._cls = cls

    def get_instance(self):        
        try:
            return self._instance
        except AttributeError:
            self._instance = self._cls()
            return self._instance

    def __call__(self):
        raise TypeError('Use Singleton.get_instance().')

    def __instancecheck__(self, inst):
        return isinstance(inst, self._cls)

Here is how to force a class to be a singleton:

1
2
3

@Singleton
class SingletonClass:
   # do whatever needed

This is how to instantiate SingletonClass() class:

`1`	`singleton_class_instance = SingletonClass.get_instance()`

The main drawback with this implementation is that it is not thread-safe.

Metaclass

An object is an instance of a class which is itself an instance of a metaclass:

class Singleton(type):
    _instances = {}
    def __call__(cls):
        if cls not in cls._instances:
            cls._instances[cls] = super(Singleton, cls).__call__()
        return cls._instances[cls]

A class can be declared singleton like follows:

1
2

class SingletonClass(metaclass=Singleton):
    # do whatever needed

While this approach copes well with inheritance, I still personally prefer the decorator approach which is handy.

Borg

The Borg approach states that the identity of the instantiated objects is not important. Borg focus on the state which must be common and shared. This allows the creation of as many instances as needed, but any of these instances which modifies the state mirrors the modification to all the remaining instances. This is why Borg is also referred as monostate design pattern:

class Borg:
   _shared_state = {}
   
   def __init__(self):
       self.__dict__ = self._shared_state

Any class which inherits from the above one becomes a Borg class itself:

class SingletonExample(Borg):

   def __init__(self, name=None):
       Borg.__init__(self)
   # Do whatever needed

Alex Martelli, the author of Borg, considers this as a design pattern of its own. While some others consider it just a special singleton implementation. I personally belong to this later group.

Conclusion

I have gone briefly gone through the main approaches Python uses to implement it. You can also design your own implementation. It is up to you the developer to decide which one is more suitable to your case.

Laravel Sanctum with Nuxt and Vuetify SPA

Sat, 07 Aug 2021 12:33:21 +0800

Sanctum allows, among other things, to authenticate an SPA via cookies using the web authentication guard. The following discussion shows how to create a single page application using Nuxt.js and VuetifyJs to connect it (sign up, sign in and logout a user) to a backend API through Laravel Sanctum. Laravel 8 is the version used below.

1. Backend

Below creates a new Laravel project named ‘server’:

`1`	`composer create-project --prefer-dist laravel/laravel server`

Once the database settings are done in server/.env file, migrations can be run:

`1`	`php artisan migrate`

At this point, Laravel Sanctum can be installed and set as per the official documentation:

# Install Sanctum
composer require laravel/sanctum

# Publish its configuration and migration files 
php artisan vendor:publish --provider="Laravel\Sanctum\SanctumServiceProvider"

# Run Sanctum migrations
php artisan migrate

# Add Sanctum's middleware to API middleware group in server/app/Http/Kernel.php file
'api' => [
    \Laravel\Sanctum\Http\Middleware\EnsureFrontendRequestsAreStateful::class,
    'throttle:api',
    \Illuminate\Routing\Middleware\SubstituteBindings::class,
],

More serious things start at this point: In routes/api.php, auth:sanctum should be used instead of the default auth:api:

1
2
3

Route::middleware('auth:sanctum')->get('/user', function (Request $request) {
    return $request->user();
});

The above route api/user is used to request the identity of the currently logged in user, i.d. after the authentication is successful.

In .env these 2 values have to be added:

# Used in config/session.php
SESSION_DOMAIN=localhost

# Used in config/sanctum.php. This should match our client
SANCTUM_STATEFUL_DOMAINS=localhost:3000

Browsers do not allow HTTP requests between different domains. With CORS, a server can agree about which client it can receive requests. This is done in Laravel in config/cors.php file by setting the following key to true:

`1`	`'supports_credentials' => true,`

It would be nice to create a user in the database at this stage. This can be done from the command line quite fast:

php artisan tinker
Psy Shell v0.10.8 (PHP 7.4.22 — cli) by Justin Hileman
>>> $user = new App\Models\User;
=> App\Models\User {#3391}
>>> $user->name = 'begueradj';
=> "begueradj"
>>> $user->email = 'billal@begueradj.com';
=> "begueradj@gmail.com"
>>> $user->password = bcrypt('begueradj');
=> "$2y$10$vXdKSDE9Qud/cy4urMrRieeUED2kNSnmZLGfMx3xZls7.69FtFCUe"
>>> $user->save();
=> true

Next, 2 controllers can be created to login and to register a user:

1.1. LoginController

`1`	`php artisan make:controller LoginController`

The below code is not perfect and does not comply with best practices but the goal is to just make things function:

<?php

namespace App\Http\Controllers;

use Illuminate\Http\Request;
use Auth;

class LoginController extends Controller
{
    public function login(Request $request)
    {
        $request->validate([
            'email' => ['required'],
            'password' => ['required']
        ]);

        if (Auth::attempt($request->only('email', 'password'))) {
            return response()->json(Auth::user(), 200);
        }

        /* throw ValidationException::withMessages([
            'email' => ['The provided credentials are incorrect.']
        ]); */
    }

    public function logout()
    {
        Auth::logout();
    }
}

1.2 RegisterController

It could be done this way:

<?php

namespace App\Http\Controllers;

use App\Models\User;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Hash;
use Auth;

class RegisterController extends Controller
{
    public function register(Request $request)
    {
        // var_dump($request->name);
        $request->validate([
             'name' => ['required'],
             'email' => ['required', 'email', 'unique:users'],
             'password' => ['required', 'min:8', 'confirmed']
         ]);

         User::create([
             'name' => $request->name,
             'email' => $request->email,
             'password' => Hash::make($request->password)
         ]);
    }
}

The new functions can be reached out through routes/api.php:

<?php

use Illuminate\Http\Request;
use Illuminate\Support\Facades\Route;

use App\Http\Controllers\LoginController;
use App\Http\Controllers\RegisterController;

Route::middleware('auth:sanctum')->get('/user', function (Request $request) {
    return $request->user();
});

Route::post('/login', [LoginController::class, 'login']);
Route::post('/register', [RegisterController::class, 'register']);
Route::post('/logout', [LoginController::class, 'logout']);

2. Frontend

A SPA can be created using Nuxt.js. Below, VuetifyJs as well as axios are going to be used during the creation process of the client project:

`1`	`yarn create nuxt-app client`

The @nuxtjs/auth-next module can be installed with:

`1`	`yarn add --exact @nuxtjs/auth-next`

Both @nuxtjs/auth-next and @nuxtjs/axios should be declared in nuxt.config.js:

modules: [
  '@nuxtjs/axios',
  '@nuxtjs/auth-next'
]

The settings for both modules are as follows:

axios: {
  /**
    When issuing a request to baseURL that needs to pass authentication headers to
    the backend, 'credentials' should be set to 'true'
  */
  credentials: true, // default value of withCredentials is fale
  
  // This is where to hit the server
  baseUrl: 'http://localhost:8000'
},

And:

 1auth: {
 2  redirect: {
 3    login: '/login',
 4    logout: '/',
 5    callback: '/login',
 6    home: '/'
 7  },
 8  strategies: {
 9    laravelSanctum: {
10        provider: 'laravel/sanctum',
11        url: 'http://localhost:8000',
12        endpoints: {
13          login: { url: '/api/login', method: 'post' }
14        },
15        user: {
16          property: false
17        },
18        tokenRequired: false,
19        tokenType: false
20      }
21  },
22  localStorage: false
23},

Lines 2 - 7 can be removed as that’s already the default behavior of nuxt auth package.

If the project is mainly admin focused with only few public pages, the authenticaion module can be set globally:

1
2
3

router: {
  middleware: ['auth']
},

And later it can be disabled on indidual public pages (routes, to be exact) as follows:

1
2
3

export default {
  auth: false
}

 2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 3738 394041 42 43 44 45 46 47 48 49 50 51 52 53 545556 57 58 59 6061 62 63 64 65 6667 1<template> <v-container fill-height> <v-row justify="center" align="center"> <v-col cols="12" sm="6"> <v-form ref="form"> <v-text-field v-model="form.email" :counter="10" label="Email" color="green" required > <v-icon slot="prepend" color="grey"> email </v-icon> </v-text-field> <v-text-field v-model="form.password" label="Password" type="password" color="green" required > <v-icon slot="prepend" color="grey"> lock </v-icon> </v-text-field> <v-btn color="blue-grey" class="ml-0" @click="login"> Login </v-btn> </v-form> </v-col> </v-row> </v-container> class="o"></template> class="o"><script> class="kr">export default { name: 'Login', data () { return { form: { email: '', password: '' } } }, methods: { async login () { // this is managed automatically in the background class="c1">      // await this.$axios.$get('/sanctum/csrf-cookie') class="c1">      try { let response = await this.$auth.loginWith('laravelSanctum', { data: this.form }) // console.log(response) class="c1">        this.$router.push('/dashboard') } catch (err) { console.log(err) } } } class="p">} class="o"></script>

Hence this component can be used in the login route:

<template>
  <login />
</template>
<script>
import Login from '~/components/authentication/Login.vue'
export default {
  components: {
    Login
  }
}
</script>

The output should look as follows:

2.2. Register component and route

The component should provide a user interface to populate the user migration file of the backend server:

 1<template>
 2  <v-container fill-height>
 3    <v-row justify="center" align="center">
 4      <v-col cols="12" sm="6">
 5        <v-form ref="form">
 6          <v-text-field
 7            v-model="form.name"
 8            :counter="10"
 9            label="Name"
10            color="green"
11            required
12          >
13            <v-icon slot="prepend" color="grey">
14              account_circle
15            </v-icon>
16          </v-text-field>
17
18          <v-text-field
19            v-model="form.email"
20            :counter="10"
21            label="Email"
22            color="green"
23            required
24          >
25            <v-icon slot="prepend" color="grey">
26              email
27            </v-icon>
28          </v-text-field>
29
30          <v-text-field
31            v-model="form.password"
32            label="Password"
33            type="password"
34            color="green"
35            required
36          >
37            <v-icon slot="prepend" color="grey">
38              lock
39            </v-icon>
40          </v-text-field>
41
42          <v-text-field
43            v-model="form.password_confirmation"
44            label="Confirm password"
45            type="password"
46            color="green"
47            required
48          >
49            <v-icon slot="prepend" color="grey">
50              lock
51            </v-icon>
52          </v-text-field>
53
54          <v-btn color="blue-grey" class="ml-0" @click="login">
55            Register
56          </v-btn>
57        </v-form>
58      </v-col>
59    </v-row>
60  </v-container>
61</template>
62
63<script>
64export default {
65  name: 'Register',
66  data () {
67    return {
68      form: {
69        name: '',
70        email: '',
71        password: '',
72        password_confirmation: ''
73      },
74      errors: {}
75    }
76  },
77
78  methods: {
79    login () {
80      this.$axios
81        .$post('/api/register', this.form)
82        .then(function (response) {
83          console.log(response)
84        })
85        .catch(function (error) {
86          console.log(error)
87        })
88    }
89  }
90}
91</script>

The register route can use the above component:

<template>
  <register />
</template>
<script>
import Register from '~/components/authentication/Register.vue'
export default {
  auth: false,
  components: {
    Register
  }
}
</script>

Hitting the register route would result in this user interface:

2.3. Authenticated user

For the authenticate user, a Dashboard component could be created.

 1<template>
 2  <v-container fill-height>
 3    <v-row justify="center" align="center">
 4      <v-col cols="12" sm="6">
 5        Hello
 6        <span v-if="user"> {{ user.name }}</span>
 7      </v-col>
 8    </v-row>
 9    <v-row justify="center" align="center">
10      <v-col cols="12" sm="6">
11        <v-btn @click="logout">
12          Logout
13        </v-btn>
14      </v-col>
15    </v-row>
16  </v-container>
17</template>
18
19<script>
20export default {
21  name: 'Welcome',
22  data () {
23    return {
24      user: ''
25    }
26  },
27  created () {
28    this.getAuthenticatedUser()
29  },
30
31  methods: {
32    async getAuthenticatedUser () {
33      console.log('loggedIn : ' + this.$auth.loggedIn)
34      try {
35        let response = await this.$axios.$get('/api/user')
36        this.user = response
37        console.log(response.name)
38      } catch (err) {
39        console.log(err)
40      }
41    },
42    async logout () {
43      console.log('logout')
44      await this.$axios.$post('/api/logout')
45      this.$router.push('/')
46    }
47  }
48}
49</script>

Line 34 requests the /api/user route in server/routes/api.php declared earlier and which is guarded by Sanctum.

This component displays the name of the user who is signed in successfully to the application, and offers a logout button to click and exit.

The corresponding dashboard route can use the above component as follows:

<template>
  <v-container fill-height>
    <v-row justify="center" align="center">
      <v-col cols="12" sm="6">
        <welcome />
      </v-col>
    </v-row>
  </v-container>
</template>

<script>
import Welcome from '~/components/Welcome.vue'
export default {
  name: 'Dashboard',
  components: {
    Welcome
  }
}
</script>

Testing emit event in Vue.js using Jest and Vue Test Utils

Mon, 13 Jul 2020 12:33:21 +0800

Often we need to design a custom component which needs to communicate data with its parent. One way to achieve this is to rely on the emit event through the v-model directive. The aim of this article is to show how to test emit event using Vue Test Utils and Jest.

For this purpose, let us implement the below user interface which consists of a label Vuetify component reflecting what is being typed within the input field:

I am going to use Nuxt.js with Vuetify to simplify the task. In pages/index.vue, we have this:

<template>
  <v-container>
    <v-row no-gutters>
      <v-col cols="4" class="d-flex justify-center align-center">
        {{ firstName }}
      </v-col>
      <v-col cols="8">
        <custom-text-field v-model="firstName" />
      </v-col>
    </v-row>
  </v-container>
</template>

<script>
import CustomTextField from '@/components/CustomTextField.vue'
export default {
  components: { CustomTextField },
  data () {
    return {
      firstName: 'begueradj'
    }
  }
}
</script>

The content of components/CustomTextField.vue file we imported from within pages/index.vue is:

<template>
  <v-container>
    <v-row>
      <v-col cols="12">
        <v-text-field v-model="firstName" />
      </v-col>
    </v-row>
  </v-container>
</template>

<script>
export default {
  name: 'CustomTextField',
  props: {
    value: {
      type: String,
      default: ''
    }
  },
  computed: {
    firstName: {
      get () {
        return this.value
      },
      set (value) {
        this.emitFirstName(value)
      }
    }
  },
  methods: {
    emitFirstName (value) {
      return this.$emit('input', value)
    }
  }

}
</script>

Above, we use a computed property to track value changes which should be emitted right away through emitFirstName() method. That is the less cumbersome and most efficient way that allows us to easy test the emit event:

import { mount } from '@vue/test-utils'
import CustomTextField from '@/components/CustomTextField.vue'

describe('CustomTextField', () => {
  test('it emits expected data', () => {
    const wrapper = mount(CustomTextField)
    expect(wrapper.exists()).toBeTruthy()
    wrapper.vm.emitFirstName('hello') // We emit data
    expect(wrapper.emitted().input[0]).toEqual(['hello']) // Data is emitted with expected value 
  })
})

The test will pass:

begueradj@begueradj:~/Development/Nuxt/nv1$ yarn test test/CustomTextField.spec.js 
yarn run v1.22.4
$ jest test/CustomTextField.spec.js
 PASS  test/CustomTextField.spec.js
  Logo
    ✓ is a Vue instance (48ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        2.226s
Ran all test suites matching /test\/CustomTextField.spec.js/i.
Done in 3.19s.

For the test to run, there are dependencies to install and settings to put in place. You will find the related demo on this Github repository.

Note the presence of test/setup.js file as recommended on Vuetify official documentation:

import Vue from 'vue'
import Vuetify from 'vuetify'

Vue.use(Vuetify)

How to use Moment.js with AdonisJs

Fri, 19 Apr 2019 12:33:21 +0800

Preamble

I decided to write this article following a question published yesterday on the AdonisJs community forum solliciting if there is a tutorial that shows how to use Moment.js in AdonisJs. I answered to that question yesterday, but I would like to share the information with others here.

Do not try to install Moment.js because Adonis.js uses it internally. This means you can take any Moment.js instance and apply on it any Moment.js method you want.

Server side

We focus only on the server side article, and we will work on the same example the question provides: a Student entity. First we create an AdonisJs application in the form of an API (but what follows will also work if you prefer to use the MVC instance of AdonisJs):

`1`	`adonis new server --api-only`

Create a MySQL database (or choose an other RDBMS if you want):

`1`	`create database test_momentjs`

Then edit server/.env file with the appropriate MySQL credentials. Now we start to create a students migration file:

`1`	`adonis make:migration students`

Open the generated file located in server/database/migrations and write this:

'use strict'

/** @type {import('@adonisjs/lucid/src/Schema')} */
const Schema = use('Schema')

class StudentsSchema extends Schema {
  up () {
    this.create('students', (table) => {
      table.string('id', 30).primary()
      table.string('name', 30).notNullable()
      table.date('dob').notNullable()
    })
  }

  down () {
    this.drop('students')
  }
}

module.exports = StudentsSchema

The field in which we are interested here is the dob, acronym for date of birth. Next create the Student model:

`1`	`adonis make:model Student`

Go to server/app/Models/Student.js and copy paste this code:

'use strict'

/** @type {typeof import('@adonisjs/lucid/src/Lucid/Model')} */
const Model = use('Model')

class Student extends Model {

  static get createdAtColumn() {
    return null
  }

  static get updatedAtColumn() {
    return null
  }

  static get dates() {
    return super.dates.concat(['dob'])
  }
  
  static castDates(field, value) {
    if (field === 'dob') {
      return value.format('DD-MM-YYYY')
    }
  }

  getName(name) {
    return name.toLowerCase()
  }
}

module.exports = Student

Everything is played on around this model, so let us break it down:

First, I let AdonisJs know that the corresponding migration file to this model is not implementing the default created_at and updated_at table columns.
We let AdonisJs know that dob is a date field by concatinating it to the dates object within the static dates() method:

1
2
3

static get dates() {
       return super.dates.concat(['dob'])
}

I ask AdonisJs to display me this date field in the format I am used to at French universities in the static castDates() method:

static castDates(field, value) {
       if (field === 'dob') {
          return value.format('DD-MM-YYYY')
       }
}

Do not worry about the effect of castDates(): it has no effect on how the dates are actually stored in the database, all what it does is to format the dates in the shape you want to display it for the client.

Of course we need a controller to handle our HTTP requests and talk with the model we just created above:

`1`	`adonis make:controller Student --type http`

Open server/app/Controllers/Http/StudentController.js and copy paste this code:

'use strict'

const Student = use('App/Models/Student')

class StudentController {

  async store({ request, response }) {
    const data = request.post()
    const student = new Student()
    const keys = Object.keys(data)
    keys.forEach((key) => { student[key] = data[key] })
    await student.save()
    response.status(200)
  }

  async show({ response, params }) {
    try {
      const student = await Student.find(params.id)
      return student.toJSON()
    } catch(e) {
      console.log(e.message)
    }
  }
}

module.exports = StudentController

Basically we are interested only in saving instances of the Student entity and fetching one of them given his id for the client so that we can check if our dob is formatted the way we asked AdonisJs to handle it. You can either use curl to communicate with this server or use my demo repository I created yesterday for this purpose.

Of course, do not forget to show the HTTP requests where to go to be satisfied when they hit the server. For this, you need to open server/start/routes.js file and type this:

1
2

Route.resource('students', 'StudentController')
  .only(['store', 'show'])

This line .only(['store', 'show']) means the HTTP requests must be directed only to the store() and show() methods located in our controller above. Any other HTTP requests not intended to commincate with these methods will fail into the 404 HTTP code.

You can use my GitHub demo and play with the code at your will.

File upload from a Nuxt.js client to an AdonisJS RESTful API server using axios

Sat, 12 Jan 2019 12:33:21 +0800

Preamble

AdonisJs documentation shows how to upload files to the server using the HTML5 <form> element. But there are cases where axios comes more handy. So let us see how to upload files from a Nuxt.js client application to an AdonisJS RESTful API server with axios. I shared this project on my Github profile.

Project setup

In my demo, the client and server code are set apart. My client application is handled by Nuxt.js:

`1`	`yarn create nuxt-app client`

and the server by AdonisJs:

`1`	`adonis new server --api-only`

On my Github project, I use Vuetify.js and Eslint.

Nuxt.js client code

Here is where the main code is located:

components/
├── FileUpload.vue
└── MainPage.vue
pages/
└── index.vue
layouts/
└── default.vue

And this is the simple user interface I want to create:

Basically I like to have MainPage.vue component to wrap all my other components; in this case I need only the FileUpload.vue component:

<template>
  <file-upload />
</template>

<script>
import FileUpload from '@/components/FileUpload.vue'

export default {
  name: 'MainPage',
  components: {
    FileUpload
  }
}
</script>

The essential part of the code is what the FileUpload.vue component contains. So I show here the full code and then will explain the main points:

 1<template>
 2  <v-container
 3    grid-list-md
 4    text-xs-center
 5    fill-height>
 6    <v-layout
 7      row
 8      wrap
 9      align-center>
10      <v-flex
11        xs6
12        offset-xs3>
13        <v-text-field
14          v-model="photoName"
15          name="photo"
16          outline
17          background-color="blue"
18          color="blue"
19          label="Select image"
20          append-icon="attach_file"
21          @click="selectImage"/>
22        <input
23          ref="image"
24          class="hide-input"
25          type="file"
26          accept="image/*"
27          @change="imageSelected">
28        <v-btn
29          class="upload-button"
30          color="indigo"
31          @click="upload_photo">
32          Upload
33          <v-icon
34            right
35            color="white">
36            cloud_upload
37          </v-icon>
38        </v-btn>
39      </v-flex>
40    </v-layout>
41  </v-container>
42</template>
43
44<script>
45export default {
46  name: 'FileUpload',
47  data: () =>({
48    photo: '',
49    photoName: ''
50  }),
51  methods: {
52    selectImage() {
53      this.photo = this.$refs.image.click()
54    },
55    imageSelected(e) {
56      this.$emit('input', e.target.files[0])
57      this.photo = this.$refs.image.files[0]
58      this.photoName = this.photo.name
59    },
60    async upload_photo() {
61      let formData = new FormData()
62      formData.append('file',  this.photo)
63      let url = 'http://127.0.0.1:3333/upload'
64      let config = {
65	headers: {
66          'content-type': 'multipart/form-data'
67	}
68      }
69      await this.$axios({
70      	method: 'post',
71      	url: url,
72      	data:  formData,
73      	config: config
74      })
75      
76    }
77  }
78}
79</script>
80
81<style scoped>
82.hide-input {
83    display: none;
84}
85*{
86    text-transform: none !important;
87}
88.upload-button {
89    border-radius: 50px;
90    color: white;
91}
92</style>

As Vuetify.js which I am using here (listed in the dependencies of the project) does not have a component which behaves as the HTML5 <input> element, I need to hide this later one when displaying the <v-text-field /> component. This is the traditional simple but efficient trick usually used in this case; then we trigger a click on this hidden file input as follows:

1
2
3

imageSelected(e) {
  this.photo = this.$refs.image.files[0]
},

My input file element is referenced with image, that is why we need to look to the references available in this DOM template with it:

<input
  ref="image"
  class="hide-input"
  type="file"
  accept="image/*"
  @change="imageSelected">

In my particular case, I am interested in uploading images only, that is why I set accept="image/*". The main thing not to forget in the AJAX request is to declare the content-type header. I think the rest of the code is self explanatory:

async upload_photo() {
  let formData = new FormData()
  formData.append('file',  this.photo)
  let url = 'http://127.0.0.1:3333/upload' // This is the endpoint of my REST API on the server 
  // below is equivalent to the enctype="multipart/form-data" we use in the <form> element
  let config = {
    headers: {
      'content-type': 'multipart/form-data'
    }
  }
  await this.$axios({
    method: 'post',
    url: url,
    data:  formData,
    config: config
  })
}

If you are a one-liner, the asynchronous code above (the await part) can be written concisely as follows:

`1`	`await this.$axios.$post(url, formData, config)`

AdonisJs REST API server code

On the server, I first set the endpoint in start/routes.js:

`1`	`Route.post('/upload', 'PhotoController.upload')`

Then I created the corresponding controller:

`1`	`adonis make:controller PhotoController`

Inside this controller (app/Controllers/Http/PhotoController.js), I received the object File() sent by the axios POST request in my client code by inspecting the request object of AdonisJs: const photo = request.file('file'). Note that I use file to reference its key in the FormData() object which contains it. Below I rename the file with the current time of the server’s machine:

'use strict'

const Helpers = use('Helpers')

class PhotoController {
  async upload( {request, response} ) {
    const photo = request.file('file')
    await photo.move(Helpers.tmpPath('photos'), {
      name: new Date().getTime() +'.'+avatar.subtype,
      overwrite: true
    })
  }
}

module.exports = PhotoController

If you upload a photo, you will find it on the server, precisely in the folder /tmp/photos. Note that you have to enable CORS for the requests to be acceped. You can do that by setting cors: true in config/cors.js file. For larger images, set the limit value of maxSize to whatever you want in /config/bodyParser.js which is 20mb by default.

Nuxt.js deployment on Gitlab

Sun, 18 Nov 2018 12:33:21 +0800

How to deploy your Nuxt.js application on Gitlab for continuous integration?

First create a Gitlab CI YAML file and name it .gitlab-ci.yml with the following commands:

image: node

before_script:
  - npm install

cache:
  paths:
    - node_modules/

pages:
  script:
    - npm run generate
  artifacts:
    paths:
      - public
  only:
    - master

Then in nuxt.config.js, add these configuration lines:

/**
* Gitlab
*/
router: {
   base: '/whatEverName/',  
},
generate: {
   dir: 'public',
},

After running the CI job, and on the project’s repository, go to Settings then Pages and click on https://namespace.gitlab.io/whatEverName to navigate your application.

How to install OpenCV 4.0 for Python3.6.5 on Ubuntu 18.04 LTS

Fri, 01 Jun 2018 12:33:21 +0800

I know OpenCV 4.0.0 will rather be released on July 2018, but the installation procedure I describe here will not change. In fact, I already I installed and tested OpenCV 4.0.0-pre on Ubuntu 18.04 (to be continued soon)

I want to share my own experience regarding this procedure. Whenever I tried this or that tutorial related to installing a recent OpenCV version for Python3.x, I stumble on errors which are not recovered by those tutorials. So all what I advice you to do is to follow the steps mentioned in the official documentation¹ take your time to inform yourself about what you are doing and be meticilous about the details²

Ubuntu 18.04 LTS ships with Python3.6 version, which is also the only default Python version available³. Obviously you do not need to install some other Python3.x version unless for a specific reason.

As I said, all what you have to do is to reproduce the steps mentioned in the official documentation. But you will encounter few problems which all fall into the same category: the documentation refers to older libariries that are now renamed and upgraded. So whenever you try to install a given library and you get this error message E: Unable to locate package some_package then you have a short list of options to do respond to your issue:

Run a custom search on Ubuntu Packages Search which will be helpful in most cases⁴.
On Terminal, type sudo apt-get install followed by the libaray’s full or half name (start with the full name first) as written on the documentation, excluding the version numbering and press Tab for autocompletition or suggestions list from which you can easily guess what is the appropriate one.
Google about it and do not hesitate to use online communities to ask.
Last but not the least: do not even try to run sudo apt-get install python-opencv: this package is incomplete⁵.

Building OpenCV from source

Nothing beats installing a library from source, so let us follow the steps mentioned in the official documentation:

Required build dependencies

1
2
3

sudo apt-get install cmake
sudo apt-get install python3.6-dev
sudo apt-get install python3-numpy

Note the documentation says python-devel but that is old, you should install python3.6-dev instead. Most packages suffixed by -devel are now re-named by -dev suffix. The truth is that it is always better to install Python related packages using pip which you can install by sudo apt-get install python3-pip and then install what you need as in this example sudo pip3 install numpy (note the difference when using apt above). But for the moment, let us continue with the Canonical instead of PyPI installation.

Do not run this command:

`1`	`sudo apt-get install build-essential`

because those libraries are already available on Ubuntu 18.04.

Let us move to the next steps, but from now on, I will comment the obsolete software mentioned in the official documentation in front of the relevant command line:

sudo apt-get install libgtk2.0-dev # gtk2-devel
sudo apt-get install libv4l-dev    # of libv4l-devel
sudo apt-get install ffmpeg        # of ffmpeg-devel
sudo apt-get install gstreamer1.0-plugins-base # gstreamer-plugins-base-devel
sudo apt-get install pkg-config libavcodec-dev libavformat-dev libswscale-dev

Optional Dependencies

Although the following consists of additional dependencies, I chose to install them instead of facing further unexpected issues. I have 1000 GB, so this does not cost me anything to install them:

sudo apt-get install libpng++-dev       # libpng-devel
sudo apt-get install libjpeg-turbo8-dev # libjpeg-turbo-devel
sudo apt-get install jasper      # jasper-devel
sudo apt-get install openexr            # openexr-devel
sudo apt-get install libtiff5 libtiff5-dev libtiff-tools # libtiff-devel
sudo apt-get install libwebp-dev        # libwebp-dev
sudo apt-get install  libtbb2 libtbb-dev libjpeg62-dev libpng16-dev libdc1394-22-dev

By now the required and most important optional dependencies are installed. It is time to install OpenCV 4.0.0 for Python 3.6. What I commented refers to the equivalent libraries mentioned on the official documentation on how to install OpenCV for an older Ubuntu system and Python (2) version.

If you do not have git install it: sudo apt-get install git. Then clone the OpenCV’s and OpenCV contrib repositories:

1
2
3

cd ~
git clone https://github.com/opencv/opencv.git
git clone https://github.com/opencv/opencv_contrib.git

Note that OpenCV and opencv_contrib must be of the same version. By default, pulling down the repositories as shown above does the job, otherwise you can check it yourself too (git checkout). Building OpenCV from Source Using CMake

Before setting up the build, give the right path to opencv_contrib`. If it is in your home directory alongside with the OpenCV’s directory then you do not have to change what is below:

cd ~/opencv
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE \
	-D CMAKE_INSTALL_PREFIX=/usr/local \
	-D INSTALL_C_EXAMPLES=ON \
	-D INSTALL_PYTHON_EXAMPLES=ON \
	-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
	-D BUILD_EXAMPLES=ON ..

When the process ends, CMake should recognize the Python3.6 interpreter as well as where numpy is as shown on my screenshot. Note that CMake, in my case, knows also about Python2.7 interpreter because I installed it myself (which thing I did not cover here, but you do not need to install it if you want simply to do what the title of this tutorial says) Always inside the opencv/build directory, run this command:

`1`	`make -j4`

I wrote -j4 because my processor has 4 cores. So you must modify this number to your needs. If you do not know the number of cores your processor has then simply run nproc on your Terminal session.

If everything is OK by now, you should be able to install OpenCV:

`1`	`sudo make install`

Enjoy !

¹.https://docs.opencv.org/3.4.1/d2/de6/tutorial_py_setup_in_ubuntu.html ².https://docs.opencv.org/3.4.1/d7/d9f/tutorial_linux_install.html ³.https://wiki.ubuntu.com/Python/Python36Transition ⁴.https://packages.ubuntu.com/
⁵.https://www.begueradj.com/installing-opencv-3.2.0-for-python-3.5.2-on-ubuntu-16.04.2-lts.html

Cryptojacking: cryptography in service of the Devil

Sun, 07 Jan 2018 12:33:21 +0800

A new revenue stream has been implemented by the end of the last year. Maybe cryptojacking did not get yet the hype it deserves but it has been already widely used. Most Internet consumers are not aware of it though its intentions are far from being innocent. It may also generate inequalities among Internet users.

Definition

Cryptojacking is about using the user’s device processor, with or without his consent, to mine cryptocurrency when visiting a given webpage which is not necessarily compromised but can do that on purpose. It starts instantly upon visiting a cryptocurrency mining webpage. In its most nefarious form, cryptojacking does not require permission or any other form of effort from the user to be run. In other words, cryptojacking consists in hijacking the user’s devices (computer, smartphone, tablet …etc) to pilfer digital profit.

Background

It all started on September 2017, when a company called Coinhive¹ devised a mining component for the Monero (coin equivalent in Esperanto) cryptocurrency that a webpage can hide in a form of a JavaScript script that looks as follows:

<script src="https://coinhive.com/lib/coinhive.min.js"></script>
<script>
        var miner = new CoinHive.User('<site-key>', 'john-doe');
        miner.start();
</script>

The motivation behind Coinhive’s product was to offer an alternative income stream for websites to ads which suffer from their own security pitfalls. This company offers a flexible JavaScript API to play with the mining component at will. Hence PirateBay is the first website used this API to raise funds²:

As you may have noticed we are testing a Monero javascript miner.
This is only a test. We really want to get rid of all the ads. But we also need enough money to keep the site running.
Let us know what you think in the comments. Do you want ads or do you want to give away a few of your CPU cycles every time you visit the site?

Personally, I see such an approach equivalent to when you ask someone the permission to rape him. That said, I remember I read somewhere else that most PirateBay users accepted an in-browser mining to reduce adds. And since that, copycats have been cropping up to claim funds raising or reducing adds. Nefarious attackers can mine digital money for themselves by compromising a given website to take advantage of its traffic³. There is even a Google Chrome extension⁴ which is removed from the Web Store for mining cryptocurrency in the background without the knowledge and consent of its users.

POW CAPTCHA

To prevent spamming activities, website provide a challenge response test in the form of CAPTCHA to be solve by the visitor to check if that is a human or a machine. But nowadays, most probably you have already encountered a website which relies on the proof of work CAPCHA. The concept of POW CAPTCHA⁵ consists in forcing your CPU to solve a mathematical puzzle in the form of a hashing algorithm for the Monero Blockchain.

This is how POW CAPTCHA looks like:

While this technique makes your surfing easier and flatters your laziness in that you do not need to solve the CAPTCHA manually by yourself, you pay the website by letting it exploit your CPU and raise your electrical bills. Of course, depending on the speed and power of your machine, you may wait more or less than other visitors to be able to surf on the website in question.

Danger

This technique is more insidious than the most common malicious JavaScript attacks such as drive-by download ones which often lead to sneak nefarious software into the user’s machine. In practice, cryptojacking is mostly intentional and secretive.

Running several cryptojacked webpages may have dramatic consequences emanating from an overwhelmed machine’s processor: that can range from interrupted work to damaging the hardware (as it is the case with the infamous Android trojan called Loapi⁶) passing by data loss or a glitch in an organization’s network. There is a simpler yet effective impact to frequently land on cryptojacking webpages: that will increase your electric bills as the CPU of your machine is “forced” to work more.

Prevention

Ad blockers can protect against cryptojacking. There is an extension called No Coin for the Google Chrome users. The last version of Opera (50)⁷ ships with a built-in cryptojacking blocker. It is worth to mention that as soon as the user closes the tab of the cryptojacking webpage, the CPU cycles inherent to it and used for mining are freed (but this may not always the case. Although lot of scanners already block Coinhive cryptojacking and categorize it as malware, attackers can always find a way to circumvent them (using obfuscated code, for instance) or by hosting their own mining intermediary for JavaScript components to call back to.

Note

POW CAPTCHA will prevent spamming activities because these later ones are done rather by computer botnet⁸, so the attacker does not worry about how much CPU power is taken from the machines under his control. On the other hand, cryptojacking compromises the browsing user experience, and maybe one day everybody will have to buy a more expensive computer in order to surf on the Internet, or have to pay more electrical bills to read news online instead of paying for the conventional membership. Definitely, given our nature as humans, cryptojacking may be a source to nourish inequality among Internet consumers.

^{¹ coinhive.com}
^{² https://thepiratebay.org/blog}
^{³ https://www.washingtonpost.com/news/the-switch/wp/2017/10/13/hackers-have-turned-politifacts-website-into-a-trap-for-your-pc/}
^{⁴ https://gadgets.ndtv.com/apps/news/archive-poster-google-chrome-extension-cryptocurrency-mining-1794930}
^{⁵ https://www.cs.utexas.edu/~seang/blog/proof-of-work-captcha.html}
^{⁶ https://www.kaspersky.com/blog/loapi-trojan/20510/}
^{⁷ https://techcrunch.com/2018/01/03/opera-now-protects-you-from-cryptojacking-attacks/}
^{⁸ https://news.ycombinator.com/item?id=7944540}

Installing Selenium drivers on Ubuntu

Sun, 15 Oct 2017 12:33:21 +0800

Here is a common exception raised when trying to invoke selenium’s webriver component for browser automation launching. Here is how to fix it.

For a professional need, I installed selenium testing framework on Ubuntu 17.04 LTS within a virtual environment I created using the recommended method I found on the documentation:

`1`	`pip install selenium`

Chrome

I then tried to launch my website on Chrome:

from selenium import webdriver


driver = webdriver.Chrome()
driver.get("https://www.begueradj.com")

I have been slapped by this exception:

Traceback (most recent call last):
  File "/home/begueradj/sel/lib/python3.5/site-packages/selenium/webdriver/common/service.py", line 74, in start
    stdout=self.log_file, stderr=self.log_file)
  File "/usr/lib/python3.5/subprocess.py", line 947, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.5/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'chromedriver'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/begueradj/sel/lib/python3.5/site-packages/selenium/webdriver/chrome/webdriver.py", line 62, in __init__
    self.service.start()
  File "/home/begueradj/sel/lib/python3.5/site-packages/selenium/webdriver/common/service.py", line 81, in start
    os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

As I usual, I read the error message, and found the last line interesting:

`1`	`selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH.`

As a robot, I googled about that weird “chromedriver” and landed on this:

WebDriver is an open source tool for automated testing of webapps across many browsers. It provides capabilities for navigating to web pages, user input, JavaScript execution, and more. ChromeDriver is a standalone server which implements WebDriver’s wire protocol for Chromium. ChromeDriver is available for Chrome on Android and Chrome on Desktop (Mac, Linux, Windows and ChromeOS).

So I obeyed to the error message I got above by downloading the latest version of ChromeDriver, uncompressed the executable file and set a path to it like this:

`1`	`export PATH=$PATH:/home/begueradj/Downloads/chromedriver`

Firefox

Even if I do not use Mozilla Firefox on Ubuntu 17.04 and Ubuntu 16.10 for speed performance reasons, I was curious and tried to achieve the same goal I did with Chrome previously:

from selenium import webdriver


driver = webdriver.Firefox()
driver.get("https://www.begueradj.com")

A similar exception has been raised:

Traceback (most recent call last):
  File "/home/begueradj/sel/lib/python3.5/site-packages/selenium/webdriver/common/service.py", line 74, in start
    stdout=self.log_file, stderr=self.log_file)
  File "/usr/lib/python3.5/subprocess.py", line 947, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.5/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'geckodriver'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "so.py", line 4, in <module>
    driver = webdriver.Firefox()
  File "/home/begueradj/sel/lib/python3.5/site-packages/selenium/webdriver/firefox/webdriver.py", line 144, in __init__
    self.service.start()
  File "/home/begueradj/sel/lib/python3.5/site-packages/selenium/webdriver/common/service.py", line 81, in start
    os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable needs to be in PATH.

How to fix this issue is highlighted by the last line of the above error message:

`1`	`selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable needs to be in PATH.`

As previously, I googled about the weird ‘geckodriver’ and found that it is a web browser engine used most noticeably the Firefox web browser. So I simply downloaded geckodriver, uncompressed it and defined a path to the executable file:

`1`	`export PATH=$PATH:/home/begueradj/Downloads/geckodriver`

I believe you can proceed similarly whatever your favorite web browser is.

Do not make a direct use of function parameters

Mon, 04 Sep 2017 12:33:21 +0800

Today, I have answered a question on Code Review website. The OP’s code sample itself was simple, but a user commented my answer stating that he never heard about function’s parameters should not be used directly.

Following my answer to this question, I wrote this: “Whatever the programming language you use, do not use the parameters inside your function” and I explained what I meant:

1
2
3

def my_function(param):
   par = param
   # Now work with "par" instead of "param"

A user was curious about the last section of my thread and commented: “Is your point on ‘do not use function params directly, no matter what language’ (paraphrased) definitely correct? I’m not doubting you, just haven’t seen this before with Python”.

Of course, with the low reputation I have on that website, I could not respond to the comments in question without justifying my argumentation with a more trusted and official resource. This has been hard for me to remember where I exactly read this information. Actually this is even harder to justify now that I do not see, AFAIK, where this is written other than in a singe book that made my day.

The book in question is called: “Code Complete: A Practical Handbook of Software Construction*”. On its Chapter 7, section 7.5, Steve McConnell has written: Don’t use routine parameters as working variables It’s dangerous to use the parameters passed to a routine as working variables. Use local variables instead*. The author has provided a Java sample code similar to the one of my answer:

 int Sample( int inputVal ) {
    inputVal = inputVal * CurrentMultiplier( inputVal );
    inputVal = inputVal + CurrentAdder( inputVal );
    ...
   return inputVal;
}

And I wanted to share the experience and text of the author about it:

In this code fragment, inputVal is misleading because by the time execution reaches the last line, inputVal no longer contains the input value; it contains a computed value based in part on the input value, and it is therefore misnamed. If you later need to modify the routine to use the original input value in some other place, you’ll probably use inputVal and assume that it contains the original input value when it actually doesn’t.

And the solution provided in Complete Code is exactly what my memory reproduced through my answer:

 int Sample( int inputVal ) {
    int workingVal = inputVal;
    workingVal = workingVal * CurrentMultiplier( workingVal );
    workingVal = workingVal + CurrentAdder( workingVal );
	...
   return workingVal;
}

The author wrote a side note in which he stated this gives you the advantage of making use of inputVal if needed within the above function’s body, or elsewhere in the program. The author adds:

Introducing the new variable workingVal clarifies the role of inputVal and eliminates the chance of erroneously using inputVal at the wrong time.
Assigning the input value to a working variable emphasizes where the value comes from.

If you still are not convinced of the author’s experience and advice, let me share with you a practical must use case of this rule, by creating a primitive tkinter based GUI that disaplays a simple button:

import tkinter as tk

class Demo(tk.Frame):
   def __init__(self, master):
       tk.Frame.__init__(self, master)
       self.create_widgets()
       
   def create_widgets(self):
       self.button = tk.Button(master, text='Demo')
       self.button.pack()

if __name__ == '__main__':
   root = tk.Tk()
   d = Demo(root)
   root.mainloop()

In the initalizer, I ignored by purpose our magic rule regarding the master parameter. The compiler triggers this exception: NameError: name 'master' is not defined .

Let us re-write the same program and be compliant to our rule:

import tkinter as tk

class Demo(tk.Frame):
   def __init__(self, master):
       self.master = master
       tk.Frame.__init__(self, self.master)
       self.create_widgets()
       
   def create_widgets(self):
       self.button = tk.Button(self.master, text='Demo')
       self.button.pack()

if __name__ == '__main__':
   root = tk.Tk()
   d = Demo(root)
   root.mainloop()

Even if this situation is, maybe, not the best to explain the importance of this rule as it involves class variables, the rule is more practical than it seems. And for sure you can find out many other practical situations where this rule must be considered and respected.

Tkinter best practices

Tue, 27 Jun 2017 12:33:21 +0800

Tkinter is the standard Python interface to Tk/Tcl GUI toolkit. Programmers often wonder how to structure their tkinter program. In this article, I am going to answer to a question posted on StackOverflow which contributions drift away from some of software engineering fundamentals.

I see many questions posted on StackOverflow and CodeReview asking for tkinter best practices and improvements. While many of the contributions to those questions are interesting, they also teach bad and even harmful coding habits that result in scalability and maintenance issues.

On the present short article, I am mainly going to address the Best way to structure a tkinter application question asked on StackOverflow and review what is good and what is bad in the provided answers.

imports

First of all, do not use a wildcard import as stated in the accepted answer. This is important when you use Tk themed widgets. To be more clear, when you code:

1
2

from tkinter import *
from ttk import *

This will result in substituting all tkinter widgets with ttk ones; but may be you want to use specific ttk widgets only.

I agree when the accepted answer states that this “makes the code completely obvious when you are using Tkinter classes, ttk classes, or some of your own”. However I disagree with the bold text statement because you should not even dare to code a class name as Button for example. Ths is what is written in the famous Clean Code book written by Robert Cecil Martin on page 50: “not to encode the container type into the name.”

So the recommanded import is as follows:

1
2

import tikter as tk # Python 3.x
import Tkinter as Tk # Python 2.x

The main application is a class

While GUI are behind the development of OOP paradigm, there are other non OOP paradigms, such as “functional reactive programming” , to design GUIs. However, they do not have, AFAIK, serious implementations. And since you will not get disappointed by relying on OOP concepts to design a tkinter GUI, I have to reproduce the nicely worded paragraph in the accepted answer to the above linked question: This gives you a private namespace for all of your callbacks and private functions, and just generally makes it easier to organize your code. In a procedural style you have to code top-down, defining functions before using them, etc. With this method you don’t since you don’t actually create the main window until the very last step. I prefer inheriting from tk.Frame just because I typically start by creating a frame, but it is by no means necessary.

The class initializer

The secret of a beautiful tkinter GUI design lays within the main class’ initializer. Thus I give it the full attention. There are few wrong things with the accepted answer however. Especially with the class’ initializer:

class MainApplication(tk.Frame):
    def __init__(self, parent, *args, **kwargs):
        tk.Frame.__init__(self, parent, *args, **kwargs)
        self.parent = parent
		
		<create the rest of your GUI here>

The first thing I would like to highlight is the number of parameters hold by the initializer: while *args and **kwargs allow a function to handle a variable number of parameters, this initializer’s design must absolutely be avoided because parameters constitute a different level of abstraction from the function’s or class’ name. In other words, when you instantiate the “MainApplication()” class, you push the reader of your code (and yourself) to think on different levels of details: while maybe one could understand what your class is doing just from its well chosen name, he will have to think about lower level details which are the arguments¹ you have to pass to it. A suitable tkinter class should not be, at worst, monadic.

The parent parameter above is also called sometimes master. I personally prefer to call it master as there is one master widget by a tkinter application, while any widget can be a parent of an other widget. The accepted answer shows you must code:

`1`	`self.parent = parent`

but it does not explain why. Actually there are are two reasons for that. The first one is common to all situations you may encounter in any programming context in any other programming language: “Don’t use routine parameters as working variables It’s dangerous to use the parameters passed to a routine as working variables. Use local variables instead”². The second reason for this way of doing things is that it will avoid you later in stumbling in many AttributeError exception situations.

The second line of code that should come right away after the self.master = master instruction is a self.configure_gui() function³ Which only purpose is to save the general settings and configurations of your applications.

For instance, in a typical configuration function you may want to specify the size of the main window of your GUI and disable resizing it and, why not, set a custom title to your application:

def configure_gui(self):
   self.master.title("Snake game")
   self.master.geometry("500x500")
   self.master.resizable(False, False)

The advantage of this approach is that you can decide to change the setting of your application whenever you want, and most importantly, you know where you will have to do it!

An other wrong thing taught by the contributions to that question is teaching to instantiate classes and creating application widgets directly from within the initializer itself. The good practice I recommend is to use a function called create_widgets() inside which you can create the widgets of your application in case you have very widgets to deal with or, in case you have to design a rich GUI with different components, areas and surfaces, follow a top-down approach by calling other smaller functions each having its own responsibility:

1
2
3

def create_widgets(self):
   self.create_menu(...)
   self.create_game_board(...)

Last but not the least, follow the SaYa idiom and never use more than one layout manager for the same containing parent widget.

So here is a basic skeleton for coding a tkinter based GUI:

import tkinter as tk


class MainApplication(tk.Frame):

    def __init__(self, master):
        self.master = master
        tk.Frame.__init__(self, self.master)
        self.configure_gui()
        self.create_widgets()
   
   def congigure_gui(self):
       # ...
   def create_widgets(self):
       # ...

if __name__ == '__main__':
   root = tk.Tk()
   main_app =  MainApplication(root)
   root.mainloop()

^{¹ Parameters are the named variables in function signatures, and arguments are the values passed to functions.}
^{² Steve McConnell, Complete Code, page 176.}
^{³ You can choose a name of your own, but respect the naming conventions specified in PEP8.}

tkinter: the SaYa idiom

Sun, 19 Mar 2017 12:33:21 +0800

Python is a paramount programming languages. The same is to be said about user graphical interfaces. SaYa is an idiom I devised for tkinter developers to help them build less error-prone applications.

Background

This morning I landed on this question asked on StackOverflow where the OP stumbled on this error he shared through his/her comment to my answer:

`1`	`NoneType' object has no attribute 'destroy'`

The origin of the problem is that the OP created a button widget within the class initializer and positioned in the same time:

`1`	`self.widget_object = ttk.WidgetType(arguments).pack()`

When a mouse click event is performed, the callback is executed:

`1`	`self.widget_object.destroy()`

Explanation

The Python interpreter raises a TypeError exception late at runtime because it was assigned to destroy the None singleton object.

In Python, everything is an object (yes, including classes), to achieve this principle, Python forces functions without a return type to return the None object. This is the case of the 3 methods used to place widgets in tkinter: place(), pack() and grid().

This means the value of self.widget_object in the initializer is None. No wonder the interpreter can not apply destroy() method on it.

Motivation

Similar situations are frequent when developing GUIs in tkinter among newbies as well as experienced developers. That is why I created the SaYa idiom which I name using my parents’s first names first syllables to thank them for giving me the opportunity to live and a chance they were not able to afford for themselves: to study.

Statement

SaYa idiom states: When building a GUI in tkinter, always place widgets after their creation.

Advantages

An idiom has at least one reason to exist. Here are the benefits of SaYa on tkinter GUIs:

Scalability
Ease of maintenance
Narrowing down the attack surface on applications as this reduces bugs

SaYa is an idiom that proves to be benefical for all Python-tkinter developers.

Python unit testing

Mon, 13 Mar 2017 12:33:21 +0800

Introduction

I first thought to write about TDD (Test-Driven Development), but given the list of reminders I have to emphasize and which would lead to a very lengthy article, I decided to shrink the work into smaller edible articles.

You can find a lots of documentation about unit testing with Python on the Internet, so why am I adding this brick? That is not your business. But landing on this page will be useful for you as I found highly ranked -by Google- online resources discussing Python unit testing where the authors either mislead you at best or misinform you, not because they do not know but because they do not give importance to details.Yes, as a programmer, your job is to be obsessed by details.

History

In the old days, when computers were slow, very simple and short programs were taking dozens of minutes to build. In those days, the days of ENIAC and EDVAC, programmers were used to take long hours trying to figure out what went wrong. It was the era of debugging. Yes, debugging is the father of all advanced testing techniques you know today. During the 1970’s, testing became a distinct idea from debugging when computers became faster thanks to IBM who released, by the start of 1970’s, the first computer to use a semiconductor memory allowing programmers to develop some primitive but yet interesting video games such as Atari 2600 by the end of that decade.

Nowadays, unit testing is already an old concept which gave birth to several grand children such as TDD, BDD and ATDD; but do not be surprised to hear some of your development colleagues think of it as a new one.

On the coming Summer holidays, you may read The Growth of Software Testing by D. Gelperin, B. Hetzel who covered the history of testing at best.

What is unit testing?

Unit testing is all about moving through the small code units of your application to see if, given certain circumstances and conditions, they respond as expected to fulfill the desired functionality. In plainer English, unit testing is about hunting for eventual errors that may lay in your program functions. If unit tests do not show errors in your code, it does not mean there are no errors or that your program is correct: they just are, when well written, a paramount milestone that help you to product a good quality software by reducing the number of errors. This means unit tests are not a means by themselves but are a way that helps to walk safely to the end.

What to test?

Obviously, you have to test your smallest code units. The smallest code units you have are not classes but the functions they wrap. If your code is not organized in terms of classes or functions then you should shift your career from a developer to a dish washer in the nearest restaurant to your living place.

After you refactor a given function without changing its functionality and you found out that the corresponding unit test no longer passes then you can fairly conclude that your unit test is wrong because unit tests are there to confirm or infirm functionality not implementation details.

How to write a unit test?

There are many details to talk about, but I am not going to highlight the basic principles you must not forget when you plan to code a unit test in Python:

Your test class must inherit from unittest.TestCase.
Your test class should be prefixed with Test.
Run the test module by calling unittest.main().
The test modules must trail with _test.
The test functions should be prefixed by test_.
Last but not the least, all unit tests must be independent from each others.

Keep this in mind and let us work on an example: the present tutorial is the first step to write a primitive calculator with a graphical user interface to perform addition, subtraction, multiplication and division of integer and float numbers.

First, create a virtual environment with the following hierarchy:

.
├── calculator.py
├── __init__.py
└── tests
    ├── __init__.py
    └── unit
        ├── calculator_test.py
        └── __init__.py

The convention when programming in Python is to gather your tests within a package called tests. As a sub-package, I created unit: this is a good practice as with real projects you will have to run other tests that unit tests.

As a starting point, let us focus on the division operation only:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import logging

class CalculatorCore:
   """Basic calculation functions: +, -, * and /
   For the moment, I deal only with "/" operation.
   """

   def __init__(self):
       pass

   def check_operands_validity(self, *args):
       for arg in args:
           if not (isinstance(arg, float) or isinstance(arg, int)):
               return False
       return True

   def divide_two_integer_or_float_numbers(self, dividend, divisor):
       if self.check_operands_validity(dividend, divisor):
           try:
               a = dividend / divisor
           except Exception as exception:
               logging.error(exception)
               raise           
           else:
               return a                          
           return dividend / divisor
       return 0


if __name__ == '__main__':
   calculator_core = CalculatorCore()
   print(calculator_core.divide_two_integer_or_float_numbers(2,4))

Save the above code in calculator.py module.

Few comments about the above code:

I chose to name the above class by CalculatorCore instead of Calculator because I will code the CalculatorGUI class on the next tutorial.
Checking the parameters validity is done in a separate function called __check_operands_validity() to avoid code duplication in further operation.

To test if everything is Ok with our class, you have to test its different code units. Let us start by testing the functionality of the main useful function which properly performs the division using Python’s built in unittest module:

import unittest
from calculator import CalculatorCore


class TestCalculatorCore(unittest.TestCase):

   def setUp(self):
       self.calc = CalculatorCore()

   def test_divide_two_integer_or_float_numbers_returns_correct_result(self):
       self.assertEqual(3, self.calc.divide_two_integer_or_float_numbers(9,3))
	   

if __name__ == '__main__':
   unittest.main()

You observe that after I imported the necessary things, I respected the previous rules. The only new element up to this point is the setUp() method which is there to instantiate once only the class CalculatorCore. Save the above code within calculator_test.py.

In this test, I want to check if my function performs the division correctly, so I used assertEqual() for this purpose.

The simplest way to run this test is to position yourself on the level of calculator.py package and execute this command:

`1`	`python -m tests/unit/calculator_test`

Note how I wrote calculator_test without the .py extension.

The output should be this:

.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

Our output says there was only one test function implemented “Run 1” and the test has passed “OK”. The “.” reflects the number of tests which passed.

As a code tester, you must also check if your unit test fails when given a wrong input:

1
2

def test_divide_two_integer_or_float_numbers_returns_error_when_divisor_is_null(self):
       self.assertEqual(3, self.calculator_core.divide_two_integer_or_float_numbers(9,0))

Run again:

`1`	`python -m tests.unit.calculator_test`

You will get this informative output which speaks by itself:

.ERROR:root:integer division or modulo by zero
E
======================================================================
ERROR: test_divide_two_integer_or_float_numbers_returns_error_when_divisor_is_null (__main__.TestCalculatorCore)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/begueradj/projects/calculator/tests/unit/calculator_test.py", line 14, in test_divide_two_integer_or_float_numbers_returns_error_when_divisor_is_null
    self.assertEqual(3, self.calculator_core.divide_two_integer_or_float_numbers(9,0))
  File "calculator.py", line 21, in divide_two_integer_or_float_numbers
    a = dividend / divisor
ZeroDivisionError: integer division or modulo by zero

----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (errors=1)

But you can do a better test: you can create a test function to check if the ZeroDivisionError is raised. This way you can get rid of the previous function test:

import unittest
from calculator import CalculatorCore


class TestCalculatorCore(unittest.TestCase):

   def setUp(self):
       self.calculator_core = CalculatorCore()
       
   def test_divide_two_integer_or_float_numbers_returns_correct_result(self):
       self.assertEqual(3, self.calculator_core.divide_two_integer_or_float_numbers(9,3))   
    
   def test_divide_two_integer_or_float_numbers_raises_exception_when_divisor_is_zero(self):
       self.assertRaises(ZeroDivisionError, self.calculator_core.divide_two_integer_or_float_numbers, 2, 0)
     

if __name__ == '__main__':
   unittest.main()

Both test pass now and you can see the exception confirmation:

ERROR:root:integer division or modulo by zero
..
----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK

The question you may ask at this stage is whether or not there is nothing else to test. Luckily there is a good (but not enough) criteria to answer to this question: it is the notion of code coverage. For this purpose, there is a tool called coverage.py which you can install as follows:

`1`	`pip install coverage`

You will get a Terminal output similar to this one:

Collecting coverage
  Downloading coverage-4.3.4-cp35-cp35m-manylinux1_x86_64.whl (191kB)
    100% |████████████████████████████████| 194kB 1.7MB/s 
Installing collected packages: coverage
Successfully installed coverage-4.3.4

Let us see the code coverage of our program:

`1`	`coverage report -m`

It says that only 73% of our code is tested:

1
2
3

Name            Stmts   Miss  Cover   Missing
---------------------------------------------
calculator.py      22      6    73%   15, 22-24, 27-28

As a thumb of rule, code coverage must not be lower than 80%. So a closer look to our program shows we did not test check_operands_validity() function. Let us do it:

import unittest
from calculator import CalculatorCore


class TestCalculatorCore(unittest.TestCase):

   def setUp(self):
       self.calculator_core = CalculatorCore()

   def test_divide_two_integer_or_float_numbers_returns_correct_result(self):
       self.assertEqual(3, self.calculator_core.divide_two_integer_or_float_numbers(9,3))
   
    
   def test_divide_two_integer_or_float_numbers_raises_exception_when_divisor_is_zero(self):
       self.assertRaises(ZeroDivisionError, self.calculator_core.divide_two_integer_or_float_numbers, 2, 0)
   
   def test_check_operands_validity_returns_false_if_parameters_is_character(self):
       self.assertFalse(self.calculator_core.check_operands_validity('b'))   
      

if __name__ == '__main__':
   unittest.main()

What are unit tests useful for?

Errors in software can cost money, reputation damage, compromised systems by nefarious users who succeed to exploit your application’s flaws, or simply lead to people death. Unit tests are not a magic wand to eliminate once for all those errors, but it helps to significantly reduce them.

If you are drawn to code many unit tests for a given function so that to cover all its possible scenarios then this means your function is not good and you have to shrink it down to smaller code units (functions) with fine grained responsibilities. A function is there to do one thing only: remember of this principle and it will save your life.

Which tools to use?

Testing is such important that it spawned a long list of testing tools in different programming languages: TestOOB , tox, TestGears are some of the good ones. Howeve, my favorite tool is Nose. Which one you should use is a matter of personal taste.

Anti patterns

One of the highly ranked tutorials on this subject ranked by Google teaches you bad practices: for example, the author named test_primes.py. If you test your code using a tool like Nose then nothing will work because Nose identifies the test files by looking for files which suffix is _test. If the author respected the PEP 8 standard, he would not be disappointed when using Nose.
The same author injects several input values for one unique test. This is a bad practice because as soon as one assertion fails, the rest are skipped. The author should use subTest() function instead.

Conclusion

Unit tests are a fundamental concept to grasp in that they help to reduce the attack surface and bugs of your applications. I did not cover the doctest subject here because I simply do not like it, do not use it professionally and more importantly it does not add a value to what I already do using unittest, nose and mock frameworks (yeah, we will see mock in the coming tutorial and in the right moment).

The best you can read about unit testing is the famous The Art of Unit Testing by Roy Osherove. The examples provided in this book, however, are in C#. As an exercise, you can complete the program of today with the remaining mathematical operations and design their corresponding unit tests before we continue this tutorial.

Installing OpenCV 3.2.0 for Python 3.5.2 on Ubuntu 16.04.2 LTS

Sun, 29 Jan 2017 12:33:21 +0800

You will learn how to install OpenCV 3.2.0 on Ubuntu 16.04.2 LTS to work with Python 3.5.2.

Introduction

I just freshly finished the installation of Ubuntu 16.04.2 LTS on my new laptop, so I had to install the programming environments I need. As I am passionate of image processing and computer vision, I needed to re-install the latest version of OpenCV (3.2.0) to be able to use it with Python 3.5.2

The best way to work with Python libraries on a Linux platform is by relying on Python virtual environments which saves headaches caused by versions dependencies. This is why I suppose you have already installed virtualenv.

Nota bene

Do not even give a try to install the wrapper package for OpenCV python bindings:

`1`	`pip install opencv-python`

The reason is that this easy way of doing things will lead you to install a package which is not enough mature as you can not even deal with videos and can not even display an image using cv2.imshow() function. So do not be lazy and follow the below steps.

Installation steps

I am sharing with you what worked for me on my machine (64 bits):

Download OpenCV 3.2.0:

1
2

wget -O 3.2.0.zip https://github.com/Itseez/opencv/archive/3.2.0.zip
unzip 3.2.0.zip

Download SIFT and SURF OpenCV 3.2.0 implementations:

1
2

wget -O opencv_contrib.zip https://github.com/Itseez/opencv_contrib/archive/3.2.0.zip
unzip opencv_contrib.zip

Install the all the packages needed to compile a debian package:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo apt-get install build-essential cmake pkg-config

Install the libraries needed to process different video and images formats:

1
2
3

sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libxvidcore-dev libx264-dev libgtk-3-dev
sudo apt-get install libjpeg8-dev libtiff5-dev libjasper-dev libpng12-dev

Install computation libraries:

`1`	`sudo apt-get install libatlas-base-dev numpy gfortran`

Install Python 3.5 development libraries:

`1`	`sudo apt-get install python3.5-dev`

Create a virtual environment where to install OpenCV 3.2.0:

1
2
3

virtualenv cv
cd cv
source bin/activate

Setting OpenCV 3.2.0 (this takes time, be patient):

cd ~/3.2.0/
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D INSTALL_PYTHON_EXAMPLES=ON \
    -D INSTALL_C_EXAMPLES=OFF \
    -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-3.1.0/modules \
    -D PYTHON_EXECUTABLE=~/.virtualenvs/cv/bin/python \
    -D BUILD_EXAMPLES=ON ..

Compiling and installing OpenCV 3.2.0:

1
2
3

make -j4
sudo make install
sudo ldconfig

Rename cv2.cpython-35m-x86_64-linux-gnu.so to cv2.so

`1`	`sudo mv /usr/local/lib/python3.5/site-packages/cv2.cpython-35m-x86_64-linux-gnu.so /usr/local/lib/python3.5/site-packages/cv2.so`

Simlinking OpenCV3.2.0 to the cv virtual environment:

1
2

cd ~/.virtualenvs/cv/lib/python3.5/site-packages/
ln -s /usr/local/lib/python3.5/site-packages/cv2.so cv2.so

These previous steps lead me to what I wanted to get:

(cv) begueradj@h4ck-B1ll:~$ python
Python 3.5.2 (default, Nov 17 2016, 17:05:23) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cv2.__version__
'3.2.0'
>>>

JavaScript malware

Tue, 24 Jan 2017 12:33:21 +0800

What are the JavaScript malware attack techniques and how to prevent from them?

Note: I first published this as an answer on InfoSec website under a previous and now deleted profile.

Server side polymorphism

Literally meaning many shapes, polymorphism is a technique used by malware authors to evade signatures based detectors. Polymorphism is qualified as being server sided when the engine which produces several but different copies of the malware is hosted on a compromised web server (Server-Side Polymorphism: Crime-Ware as a Service Model (CaaS)). simulated metamorphic encryption generator (SMEG) version 1.0 was the first engine developed to implement the notion of polymorphism for computer viruses on the early 1990’s (Parallel analysis of polymorphic viral code using automated deduction system)

Code obfuscation

The other common feature you may find in malicious JavaScript code is that obfuscation is always used. This common factor -obfuscation- does not make even things simpler: because innocuous JavaScript code also uses obfuscation (for instance, some developers for example do not want their personal pretty JavaScript function to be understood by others as you can easily read HTML and JS pages codes). Along with server side polymorphism, code obfuscation is a widely used technique by malware authors to circumvent antivirus scanners. A myriad of techniques could be used to obfuscate JavaScript codes such as string reversing, Unicode and base 64 encoding, string splitting and document object model (DOM) interaction (Malware with your Mocha? Obfuscation and anti-emulation tricks in malicious JavaScript.).

Code unfolding

Code unfolding is the mechanism with which a new code is introduced at run time. In JavaScript, this is made concrete by invoking functions like document.write() and eval() in order to execute obfuscated portions of code and functions. (Weaknesses in Defenses Against Web-Borne Malware)

Heap spray

This attack targets mainly web browsers. The user controllable data can corrupt the heap by a remote execution code if the miscreant has compromised the user’s computer to the point he can have access to this vulnerable memory area (BuBBle: A Javascript Engine Level Countermeasure against Heap-Spraying Attacks)

Drive-by download

Drive-by download attacks consist in downloading and and executing or installing malicious programs without the user’s consent. Such attacks occur by exploiting browsers’ vulnerabilities, their add-ons or plugins such as ActiveX controls or unpatched useful software such as Acrobat Reader and Adobe Flash Player (Drive-by download attacjs: effect and detection methods, MSc Information Security)

Multi execution paths

It is possible to trigger an action only if certain conditions are fulfilled. Such circumstances could be the arrival of a given date or the existence of a file on the system on which the malware is intended to be executed. An other quick and well known example could be a denial of service attack that must be fired only if the number of the botnet’s nodes has reached a certain value. That is the notion of multi execution paths (Exploring Multiple Execution Paths for Malware Analysis)

Implicit conditionals

This technique is mainly used against dynamic approach detectors. The main idea for this process is to execute a set of instructions by hiding the condition that fires it (Weaknesses in Defenses Against Web-Borne. Malware)

Given these common features and tactics used by JaaScript malware, if you want to detect this type of malware as you asked, you need first to study the state of the art of the methods used to detect that. Various methods have been developed so as to detect web (JavaScript) malware. We can divide them into two main categories as follows:

Machine learning based classifiers

Features: HTML and JavaScript codes distinguishing features extraction. These features are then evaluated to train a machine learning for classifier generation. The premise of this approach is that malicious webpages are likely to be different from benign ones (Thesis: Effective Analysis, Characterization, and Detection of Malicious Web Pages)
Advantages: Lightweight approach, useful to deal with a bulk of websites analysis.
Drawbacks: Obsolete against obfuscated JavaScript code and totally useless against new malicious code patters or zero attacks.

Dynamic methods

Features: Based on the dynamic behavior analysis, these techniques are implemented using either proxies where a page is rendered to the visitor only after its safety is checked, or a sandboxing environment relying on honeyclients (Same thesis: Effective Analysis, Characterization, and Detection of Malicious Web Pages).
Advantages: Efficient against zero day attacks and obfuscated code.
Drawbacks: Resources and time consuming. Sandboxing environments rely on low interaction honeyclients which themselves are based on virus signatures, and thus suffer from the same disadvantages as the static methods’ ones.

What you have tried to do belongs to the first category.

Now, after you are well informed about all this, it can be useful for you to study some available tools dedicated for this purpose in order to implement your own technique. So let me mention you three important tools among so many others:

Zozzle

Zoozle relies on Bayesian classification abstract syntax tree (AST) . It is legitimately classified as mostly static web malware detector because it embeds another engine that supervises the JavaScript code execution at run time. Its authors claim that it has a very low false positive rate of 0.0003% and is able to process over one megabyte of HTML and JavaScript code per second. This tool is intended to be used as a browser plugin; its aim is to protect browsers against heap spray attack. It is time to point out how ZOZZLE operates.

How ZOZZLE operates? The following figure summarizes its core (ZOZZLE: Fast and Precise In-Browser JavaScript Malware Detection):

Extraction and labeling phase: The classifier needs training data. This data is extracted from obfuscated JavaScript code. Instead of developing an efficient de-obfuscation technique, Compile function interception calls is performed. Compile function is located in jscript.dll library. It is a smart way to obtain plain JavaScript code because it is called each time <SCRIPT> and <IFRAME> tags, or eval() and document.write() functions have been called, which thing defines also the code context. Each code context is saved on the hard drive for further analysis.
Feature selection: JavaScript AST is used to tag each labeled context code for its safety or malignancy. The features are pre-selected using this formula:

Where:

A: malicious context with feature
B: benign context with feature
C: malicious context without feature
D: benign context without feature
Classification: The Bayesian classifier is used for classification because even if it seems obsolete, in practice it gives good results and it is not time consuming.

Profiler Profiler follows the static schema to detect web malware. It combines static features analysis of HTML and JavaScript code, including unified resource locator (URL)s. Then it uses machine learning techniques to teach a classifier that decides if a webpage embeds malicious content or not. Suspicious webpages are not processed by this tool. It rather forwards them to third party technologies such as Wepawet (Prophiler: A Fast Filter for the Large-Scale Detection of Malicious Web Pages)
SpyProxy

SpyProxy follows the dynamic analysis principles. It monitors the active content of webpages within a virtual machine before deciding to render them to the visitor or not. The architecture of SpyProxy is illustrated through this figure (SpyProxy: Execution-based Detection of Malicious Web Content):

(a): The proxy performs a static analysis over the requested page. In the case it judges is likely to be malicious, if forwards it to the virtual machine. basically only pages with active content are forwarded to the virtual machine (VM).
(b): The virtual machine loads the malicious pages to monitor their activities.
(c): Only benign pages are rendered back to the proxy which forwards them in turn to the user’s browser.

Iceshield

ICESHIELD performs in-line dynamic code analysis using a set of heuristics to verify attack attempts. Its authors take an inventory of the attacks that usually target the DOM properties of a website that are performed by injecting JavaScript into the website’s source code. ICESHIELD supervises the running JavaScript code by predefining a set of rules related to functions calls and applying heuristics on them in the hope to determinate whether the script is malicious or not (IceShield: Detection and Mitigation of Malicious Websites with a Frozen DOM).

Mozilla beheads SHA-1

Tue, 17 Jan 2017 12:33:21 +0800

You have heard of SHA-1 deprecation since 2012. Mozilla finally confirms this is not a joke.

May be you are tired of hearing over and over again about SHA-1 deprecation since 2012 when Bruce Schneier announced that collision attacks may be cost around $700k to perform by 2015, which was the year where the first practical full-on collision attack was lead out by researchers who predicted a costless real world attack may occur on 2018.

Since that, Google, Microsoft and browsers manufacturers have raced the migration from SHA-1 to SHA-2. On last Tuesday, Mozilla gave the pace for this rush by being the first to display the warning Untrusted Connection on its Firefox browser to users visiting a website which SSL/TLS certificate is signed by SHA-1 instead of SHA-256.

Now you may think this paranoia makes no sens as no real world attacks exploiting this technique are known by today. My answer is that lack of paranoia is a synonym of unconsciousness: publicly, there are no known attacks, that is true, but under the hoods who knows? Also, history teaches us how MD5 which is more prone to collision attacks was equivalently used in cyber espionnage attacks against Iran through the famous Flame malware.

The SSL/TLS certificate installed for my website uses SHA-2 as this Python script says:

1
2
3

import os
os.system('openssl s_client -connect www.begueradj.com:443 < /dev/null 2>/dev/null\
          | openssl x509 -text -in /dev/stdin | grep "Signature Algorithm"')

Output:

1
2

Signature Algorithm: sha256WithRSAEncryption
Signature Algorithm: sha256WithRSAEncryption

I just wonder how much this migration would cost as, in real life, SHA-1 underpins around 35% of the digital certificates existing today resulting without mentioning the nightmares engineers are enduring regarding their unsupported applications or hardware reconfigurations to perform.

Billal BEGUERADJ

The broken window theory

Python decorators

Introduction

Back to 2005

Definition

Python functions

FCO

Nested functions

Closures

Decorators

Definition

Example

Decorators of functions with arguments

Chaining decorators

Decorators and OOP

What are decorators useful for?

Conclusion

Effective file streaming using AdonisJs

Preamble

Our stack

Our app

Project setup

Nuxt.js client code

AdonisJs REST API server code

Python singleton design pattern

Introduction

Definition

Implementation

Module

Traditional

Decoractor

Metaclass

Borg

Conclusion

Laravel Sanctum with Nuxt and Vuetify SPA

1. Backend

1.1. LoginController

1.2 RegisterController

2. Frontend

2.1. Login component and route

2.2. Register component and route

2.3. Authenticated user

Testing emit event in Vue.js using Jest and Vue Test Utils

How to use Moment.js with AdonisJs

Preamble

Server side

File upload from a Nuxt.js client to an AdonisJS RESTful API server using axios

Preamble

Project setup

Nuxt.js client code

AdonisJs REST API server code

Nuxt.js deployment on Gitlab

How to install OpenCV 4.0 for Python3.6.5 on Ubuntu 18.04 LTS

Building OpenCV from source

Required build dependencies

Optional Dependencies

Enjoy !

Cryptojacking: cryptography in service of the Devil

Definition

Background

POW CAPTCHA

Danger

Prevention

Note

Installing Selenium drivers on Ubuntu

Chrome

Firefox

Do not make a direct use of function parameters

Tkinter best practices

imports

The main application is a class

The class initializer

tkinter: the SaYa idiom

Background

Explanation

Motivation

Statement

Advantages

Python unit testing