Posts in Programming

audio processing

10 Deep Learning Resources for Audio Processing

April 19, 2019 Posted by Programming 0 thoughts on “10 Deep Learning Resources for Audio Processing”

We’ve written before about the rise of voice assistants in the IoT market. As these devices become more and more sophisticated, we can expect to see them number of voice assistant to skyrocket to 8 billion by 2023. This emerging market creates an excellent opportunity for those interested in audio processing or audio anomaly detection. In the future, there may be more emphasis put in creating valuable resources that can help up-and-coming audio processing data scientists practise their craft. As of now, there are plenty of resources available. Below are 10 resources for audio processing.

 

CS Machine Learning for Signal Processing

These are signal processing notes from a Computer Science class, but they cover the basics of the mathematical background you need to then perform other tasks. The topics include DSP Primer, Perception and Features, Principal Component Analysis, ICA and NMF, KPCA and Manifold Methods, Detection and Matched Filters, Decision theory & classifiers, Nonlinear classifiers, Classification bits and pieces, Clustering, DTW and HMMs, Missing data & dynamical models, Arrays & source separation, Underconstrained separation, Deep Learning.

 

AudioSet

You can’t train model without data. Audio Set provides millions of free sound clips.

 

MadeUpMasters

A forum user posted a comprehensive guide about deep learning with audio.

 

Speech and Audio Understanding

Another CS course filled with lecture slides. These notes focuses on auditory processing by understanding both biology and the underlying technical process.

 

Human and Machine Hearing: Extracting Meaning from Sound 

This book, written by Richard F. Lyon, will teach you how humans hear and how to build machines that respond to them. It’s worth a read for anyone involved in audio processing.

 

Aubio

“aubio is a library to extract annotations from audio signals: it provides a set of functions that take an input audio signal, and output pitch estimates, attack times (onset), beat location estimates, and other annotation tasks.”

 

LibROSA

“LibROSA is a python package for music and audio analysis. It provides the building blocks necessary to create music information retrieval systems.”

 

Urban Sound Classification with Neural Networks in Tensorflow

“This post discuss techniques of feature extraction from sound in Python using open source library Librosa and implements a Neural Network in Tensorflow to categories urban sounds, including car horns, children playing, dogs bark, and more.”

 

Audio Classification using FastAI and On-the-Fly Frequency Transforms

This article provides a more informal approach to Deep Learning and Audio Classification by introducing a practical technology called FastAI.

 

Science Wiki for Audio Processing

If those suggestions weren’t enough and you want to completely geek out on audio processing, search for audio processing in the Science Wiki.

Please follow and like us:
0

5 web development practices challenged

April 19, 2019 Posted by Programming 0 thoughts on “5 web development practices challenged”

There are many myths in the software business that have led to wrong best practices. In this post I will address 5 of these best practices and explain on which wrong assumptions they are based. I’m worried about the state of the industry, because I feel these are serious engineering mistakes and nobody is speaking up about them. Here we go:

1. Client side rendering

Based on the wrong assumption that client side rendering is faster than server side rendering we frivolous apply React and Vue. Not even the load on the servers is diminishing as escaping data for HTML or JSON is equally expensive. Rendering HTML is faster than executing JavaScript to draw the DOM. But don’t you need to send much more data when you are sending HTML? Well not really, because you can send all dependencies in separate files. It was for HTTP 1.0 that web pages were in-lining CSS and script to avoid TCP connections. In HTTP 2.0 all requested files are multiplexed over the same connection. Today we can have multiple separate files, which increases the ability to cache these files. More cached files reduces the data transfer which makes your site faster. This is also why resource bundles should be avoided.

2. Event-loop vs threading

NodeJS uses event-loops to run servers. These are said to be faster when connections need to talk to each other (avoids IPC) and for I/O intensive tasks. The reason this is true is because you are avoiding the context switching that multi-programming requires (for security, to isolate processes). And as long as you are only using a single core of your machine this non-existing IPC cost is true. But most servers in production actually have multiple processors and those have 10 or more cores each. And even when your processes can be run multiple times and run completely individually, you need to make sure you concurrency is exactly right per core (a difficult load balancing task). You also need to make sure that you are not doing any computational tasks or blocking I/O in your threads or your servers will be very slow (as your latency will spike). The threading model may be less performant in an optimal case, but in all realistic situations it will be faster as it does not require meticulous tuning.

3. Micro-services with their own database server

When you order a trip at a travel agency they have to order a seat at an airplane and then order a hotel room and a car for you. If one of these reservations fails you probably want to go somewhere else where they do have all three available for you. This problem is called “transaction support” in databases and it is solved quite elegantly. The goal of transactions is to have high data consistency and no accidental booked – but not canceled – hotels hanging around. Other consistency features that databases provide are foreign key constraints. If you are implementing micro-services with their own databases you have to either drop transaction and foreign key support, which will lead to data inconsistency, or re-implement transaction and foreign key support, a very daunting task. Both are a very bad idea, so you should stick to a single database server when implementing micro services. Everybody who tells you otherwise should be challenged to explain how to implement “two phase locking”.

4. MongoDB as a primary store

MongoDB is a NoSQL store and it is fast! It is seriously fast as long as all your data fits in RAM. What they don’t brag about is that it has low durability guarantees: it only flushes data to disk every 60 seconds. You can also tune a MySQL server to use all the RAM for indexes and data. You can even set the “innodb_flush_log_at_trx_commit” variable to zero to flush only once per second and avoid a flush at every commit. Suddenly the performance of MySQL is a lot closer to the performance of MongoDB. I wrote an article titled “Trading durability for performance without NoSQL” on how to do this in various databases. Also databases without foreign keys and table structure may seem flexible, but they come at the cost of inconsistent data that piles up in your database. I would rather have less flexibility and more consistent data in my primary store. But if you do not care about the quality of your data, then MongoDB may be great choice.

5. Database technology independence

People use a DBAL (DataBase Abstraction Layer) and/or ORM (Object Relational Mapper) to not having to write SQL or (heaven forbid) stored procedures. I have not seen this work out well for a few reasons. Developers need to be familiar with SQL to be able to write efficient queries in large systems. If you don’t know exactly what SQL is executed, because you use an ORM, you can easily make a (performance) mistake. I have also seen many very expensive algorithms that could have been replaced with a rather cheap and simple stored procedure. But apparently stored procedures are not “cool” anymore and reasons given are the database independence and that supposedly code does not belong in the database. This last thing may be somewhat true, but with some proper versioning you can achieve a lot. On the database independence I can say that the independence is almost never achieved and you are constantly paying the price, so this is really a case of YAGNI (You Aint Gonna Need It).

Please follow and like us:
0
TCP

Explaining TCP/IP Like You’re Five

April 18, 2019 Posted by Programming 0 thoughts on “Explaining TCP/IP Like You’re Five”

Before we get into the simplified explanation of Transmition Control Protocol, here’s a more technical breakdown.  TCP is a “standard that defines how to establish and maintain a network conversation via which application programs can exchange data.”

  • Working together with the IP (Intern Protocol) to form TCP/IP, TCP establishes a connection that allows the Application Layer to send packets.
  • These packets are delivered to another application by the IP layer or Network Layer
  • The size of the packets sent across the connection are regulated by TCP.
  • Lost or damaged packets are fixed by TCP and sent to the destination by the IP Layer once more.
  • The network layers in which TCP does its active work, as assigned by Open Systems Interconnection, are the Transport Layer and Sessions Layer.
  • Once communication ceases, TCP closes the connection.

 

We can rephrase the technical explanation by imagining a Medieval pizza delivery system. For the sake of simplicity and fun, let’s pretend that pizza was invented in the middle ages.

TCP/IP

Here are some facts

  1. There are two dukes(Duke Blue and Duke Red) that are currently waging war against one another.
  2. Their castles(Castle Blue and Castle Red) are on opposite ends of a ten mile stretch of land.
  3. The only road that one can take to reach one castle or the other is Duke’s Way. The road is closed and is guarded by men who only allow messengers through.

Now we’ll explain

A citizen of Red Castle(client) sends a messenger(HTTP/DNS/SMTP) to order pepperoni pizza(data) from Bobby(server), a renown pizza maker in Blue Castle.

The messenger(HTTP/DNS/SMTP) gains approval from the guards(TCP) who then open up the road(IP).

Bobby(server) receives the order from the messenger(HTTP/DNS/SMTP). Bobby makes the pizza and gives it to the messenger(HTTP/DNS/SMTP).

The messenger(HTTP/DNS/SMTP) then travels across the still open road(IP) and delivers the pizza(data) to the Red Castle citizen(client).

The guards(TCP) close the road(IP) after the messenger(HTTP/DNS/SMTP) delivers the pizza(data).

 

 

Please follow and like us:
0

The 5 worst programming languages to learn in 2019

April 18, 2019 Posted by Programming 0 thoughts on “The 5 worst programming languages to learn in 2019”

While there’s no definitive worst or best programming language, if you’re looking to learn a language that’s in demand and with an active community, some are better than others.

Codementor has compiled a list of languages struggling to attract developers or interest from employers.

The so-called “Worst Programming Languages to Learn in 2019” list is the latest in the site’s annual round-up, with Codementor stressing the list reflects languages that are a poor choice to learn as a first language.

These sorts of lists are always controversial, and one thing to bear in mind is you may not agree with the methodology used by Codementor, an online community for mentoring developers. The methodology used is explained at the end of the article.

While the worst five languages to learn are listed below, there are some other surprising callouts in the list, with some widely used and often praised languages finding their way into the worst 20.

Big names that took a dive

Kotlin

Despite Kotlin being celebrated for offering Java developers a more modern and enjoyable language to code in, Codementor says it jumped from the 18th to the 11th worst language to learn.

While Kotlin is a fully supported language for Android development, Codementor says there was only middling community engagement for the language and so-so job prospects. Combined with a slip in the language’s prospect for growth, by Codementor’s estimation, this was enough to see the Java Virtual Machine (JVM) targeted language climb the rankings of worst languages.

R

The statistical analysis language R also had a bad year by Codementor’s reckoning, with the data science-focused language climbing to 12th on the list.

Codementor attributes this poor showing to its lackluster growth last year and falling community engagement.

While R also dropped one place in this year’s RedMonk Programming Rankings, RedMonk analysts cautioned against reading too much into the slip, saying the language continued to serve a “vibrant base of analytical and data science use cases”.

Is there a Ruby renaissance?

Despite Ruby’s fortunes taking a knock of late, the language, once popular among web developers, did enjoy a small turnaround last year, according to Codementor.

The language fell three places to the 17th worst programming language to learn, which Codementor attributes to strong community engagement and job demand, enough to offset its poorer showing in Codementor’s growth and trends category.

The worst programming languages to learn in 2019 according to Codementor

1. Elm

What is it? Aimed primarily at web developers, Elm is functional language that compiles to JavaScript, and is designed to make it easier to create web apps.

Why Codementor says you shouldn’t learn it: Multiple reasons, chiefly they say it has the fourth worst community engagement, falling growth, and although the job prospects for Elm improved slightly, it was still ranked as the fifth worst language in the job-market category.

What other language surveys say: Not very much, Elm doesn’t figure in this year’s round-up of languages by Stack Overflow or RedMonk.

2. CoffeeScript

What is it? Again, primarily a web development language, CoffeeScript compiles into JavaScript and is designed to improve on JavaScript’s readability.

Why Codementor says you shouldn’t learn it: Codementor is scathing about CoffeeScript’s community engagement, naming it the worst language for engagement last year. It also says the language’s already poor growth took a nosedive over the year, arguing “CoffeeScript’s heyday is further behind it than that of other languages”. While job prospects also worsened, Codementor says there are still jobs seeking CoffeeScript skills.

What other language surveys say: Only mentioned in passing in this year’s RedMonk Programming Language Rankings, with a note that CoffeeScript is less popular than Rust. It also features in the lower echelons of the Tiobe Index of 100 popular languages.

3. Erlang

What is it? Erlang was created by the Swedish telecoms firm Ericsson almost three decades ago to help build telephony applications. Designed to support large-scale routing of telephone calls and handle faults without collapsing, it’s suited to building reliable and scalable applications.

Why Codementor says you shouldn’t learn it: Erlang saw the largest decline in community engagement over the course of the year, with interest in the language also dropping, even relative to others in its functional language niche. Ranked the fourth worst language to learn in terms of job demand, Codementor points out that while there are still more Erlang developers than jobs available, that demand for developers isn’t growing as fast as other languages.

What other language surveys say: A big caveat to Codementor’s judgement is the salary for roles associated with Erlang, with Erlang being the ninth highest-paid language and the 22nd “most loved” language according to this year’s Stack Overflow Developer Survey.

However, Erlang also experienced a precipitous drop in The Tiobe Index over the course of last year, falling from number 23 to 50.

4. Lua

What is it? A lightweight, embeddable scripting language that is commonly used in games and offers decent performance relative to other scripting languages.

Why Codementor says you shouldn’t learn it: While community engagement for Lua rose last year, it had flat growth and was ranked second worst for job demand, with Codementor saying “there are still more Lua developers than there is demand for them”.

What other language surveys say: Rated 30th in the Tiobe Index of popular programming languages.

5. Perl

What is it? Popular for web server scripting, sysadmin jobs, network programming and automating various tasks, Perl has been used since the late 1980s.

Why Codementor says you shouldn’t learn it: Worsening community engagement, declining growth, and supply outstripping demand in the job market are the primary reasons given, with Codementor saying “Perl is facing a downward trend in terms of developer interest”.

What other language surveys say: Perl has long been in the top 20 of the Tiobe Index of popular programming languages, and placed at number 14th in this April’s index, while also charting 18th in the RedMonk Programming Language Rankings for this year.

The list of worst programming languages to learn is based on Codementor ranking each language in terms of its community engagement: based on activity on GitHub, Twitter, Stack Overflow, Facebook, Freenode, and Reddit, growth: based on Google Trends and Stack Overflow Trends, and job market: with demand based on stackshare.io, techstacks.io, and CodementorX client requests and supply based on Stack Overflow’s 2018 Developer Survey and results from CodementorX’s proprietary data. Source

Please follow and like us:
0
code ownership

Developers Talk About Dealing with Code Ownership

April 17, 2019 Posted by Programming 0 thoughts on “Developers Talk About Dealing with Code Ownership”

Eons ago(2006, to be exact), Martin Fowler outlined three types of code ownership: strong code ownership, weak code ownership, and collective code ownership. Strong code ownership involves modularizing parts of code and assigning each module to a specific developer. The developer would then make changes to that module alone. Modules that are slightly dependent would require the developer to have to communicate changes made. If a developer wants to make changes to modules he/she does not own, he/she would have to send requested changes to the the developer that owns the module.

You can start to see how problems may arise with this form of code ownership. You may have a situation where the bus factor comes into play: what happens when a member of that team suddenly quits and leaves behind months’ work of undocumented code? Martin’s point was that strong ownership was deeply problematic due to the fact that communication issues could hinder development speed.

There are just too many situations where something you need to do needs changes to other people’s code. Persuading them to make the change and waiting for the change often takes so long that it leads to delays and deeper problems, this is particularly galling when the change is a simple one.

Weak code ownership loosens the restrictions a bit by allowing developers to make changes to the modules of other developers while collective code ownership does away with individual ownership altogether. Whatever type of code ownership a company chooses to employ, there will be pros and cons. Examining a Hacker News thread, I was able to get a general sense of the type of code ownership being practiced today. Here are developers’ takes on the type of code ownership that their company uses.

 

1.

In our case, teams don’t own projects/code-bases, they own “features” and “releases”.

About 67% of our code is some form of Ruby – some of it Rails, some of it custom Ruby code running in Kubernetes+Docker processing data.

The other 33% of our code is Rust – and not just service side code but Rust that has to run on multiple platforms with varying deep operating systems knowledge.

My team generally owns Features and Releases related to the Rust code because two of the three members are OS programmers by experience. That said, we don’t exclusively work on the Rust code and sometimes wander into the Ruby code base.

Similarly, other teams occasionally need to update the Rust code to generate some new data – a lot of that is pretty basic stuff and doesn’t require deep OS knowledge. So they’ll make changes there.

Since our stack is so heterogenous, it’s hard to have everyone work on everything. However, from what I know of most companies outside Apple/MSFT/Google/etc, they’re gonna fall in the “we own web services” where the only real bifurcation will be “frontend code” and then “middle-to-backend code”. Maybe throw in some apps as well.

=========

Now all of that said – the reason we went to teams owning “features” and “releases” was to push folks to not be possessive of code. Stuff changes and it changes quickly, so there’s not much use trying to isolate folks. Document gratuitously, test heavily, and refactor often.

 

2.

In my experience with several big organizations (20,000+ employees), they tend to have a home-grown application to track ownership of resources (assets) – including source code (repositories), and servers, databases, scheduled jobs (in autosys or suchlike).

Generally each of these entities is grouped into some bucket for an application / “system” / business area, and then that bucket has a few owners assigned, and also other non-ownership roles assigned (developer role can check in code to those repos, prod support role can log into the servers owned by the bucket, etc).

It’s generally heavily integrated with an LDAP directory, HR system to handle joiners and leavers, and also change management (Developers can only ssh to a production host during the change window of a change for that bucket).

These home-grown apps are generally old and clunky (written for IE6, with piecemeal improvements since then, that kind of thing), but hard to replace because they’re integrated with so many other applications.

I tend to think that this would be a good opportunity for some (startup) to do a great job of this as an off-the-shelf product, with good integrations with Linux + Windows authentication, an API, nice web app, etc – it would be very easy to do better than the in-house systems filling this niche.

 

3.

At my current workplace, most top-level subdirectories in the source tree have a primary author. That person is (nominally) “owner” of that code. If they aren’t the one making changes, they’re almost always at least the one doing the code review.

Bugs by default get routed to the person who wrote the code (presuming that we can tell which code the bug is in). If that person finds out the bug is somewhere else, the bug gets transferred to a new owner.

 

4.

I work for a large global company, and my department is building a fairly large Angular application. My team “owns” the primary project and several shared component libraries, and other teams build features for this project.

These teams are from all over the world, and many of us have never worked closely before this project began.

When we first started the project, we decided to split the UI into feature projects which are packaged and referenced in the common project. Each team owns their own feature projects; however, everyone is welcome to contribute PRs to any of the projects.

Some of our projects are shared and maintained by many teams, but we always have one team that owns each project.

 

5.

Ideally you want to restrict changes to the owners (I don’t mean hard limits) So instead of having a team responsible for a part of code, they are responsible for the product. So if a new integration is needed, the owner is a service provider and should provide the interface or service for their client. That is fairly easy to setup with Microservices. However, that is a bit harder in a monolithic application code base.

In my workplace, we use the above model for almost everything. And there is one old code base where we use Github code ownership feature to force everyone to ask for a code review from owners. But even there, it doesn’t happen very often that people from different domain need to touch a code of another domain.

And that is the second point as well. Dividing responsibilities base on domains instead of components/modules between teams make it easier as most of the time they are not going to touch a code outside their own domain.

 

6.

Everyone “owns” the code. Then there’s the guy you go to when it fails.

Helps to document things, so you at least have something to throw at people when they come asking.

 

 

Overall, collective ownership seems to be the dominant form of code ownership. The line between weak ownership and collective ownership begins to blur a bit when a code base is split into features where ownership is temporary. Even then, a subset of teams claim a collective ownership over a feature. The size of a company also seams to effect the type of ownership used. For example, a company that gives complete ownership of debugging to someone who is on call can do so if they’re company has 20 or fewer engineers. Once the number of engineers balloon in hundreds, then a collective ownership is needed to maintain productivity.

Please follow and like us:
0

Three Ways To Advance Your Career As An International Software Developer

April 17, 2019 Posted by Programming, Recruiting, Startups, Technology 0 thoughts on “Three Ways To Advance Your Career As An International Software Developer”

Here are the 3 best practices we teach to young developers in our network as they begin their journeys as international software engineers.

1. Think About Time

Do you work with a lot of 45-year-old software developers at your current company? According to a 2015 survey on Stack Overflow (via InfoWorld), the average software developer is 28.9 years old. (Data USA puts the average at 39.7.)

While there are many reasons for this, the bottom line for many developers is that the window to fully develop your professional network may be limited compared to other industries such as the legal profession, where the average age for U.S. judicial workers (including lawyers) is 46.3. 

Sure, it’s tempting to take your excellent pay to go on every available holiday and let your friends know you’re living the good life on Instagram, but it also pays to take some risks early in your career. So take risks.

Risk joining a startup, risk starting your own project or company, or risk asking someone you’d like to emulate to mentor you. As the old adage goes, “Fortune favors the brave.”

While press headlines may focus on the Zuckerbergs of the world, I know plenty of developers who were early employees at technology companies you’ve never heard of that were acquired, and who have leveraged such experiences to leapfrog their careers.

So surround yourself with people who embrace risk. You can easily spot these people and organizations because they’re the ones who are constantly investing in new experiments.

Believe me, not many developers want to join the ranks of those who thought joining my first startup was “too risky.” That company is now worth over $3 billion on the NASDAQ stock market.

I believe the greatest risk for young developers is not taking any.

2. Use IT Outsourcing As A Stepping Stone, Not A Career Destination

In emerging economies, working for an IT outsourcer can be a great place to learn basic skills and get exposure to live commercial projects, but you may not want to think of this as a long-term career prospect. That’s because IT outsourcer culture can be one of employment arbitrage and not innovation, and learning to think like an innovator can widen your career opportunities greatly.

A 2016 Deloitte survey (via The Wall Street Journal) indicated that only 21% of enterprise outsourcing contracts had any “proactive innovation” built into their service contracts, although Deloitte found that number had risen to 43% in 2018 (download required). 

Obviously many innovations come from startups, so working on a startup project could be great exposure to innovation culture and give you the opportunity to put your own unique ideas into production code.

As well, I’ve found that IT outsourcers (especially large ones) tend to work on legacy projects and in legacy languages like J2EE. This can severely limit the types of jobs you can get in the future and rob you of the experience to learn new technologies or frameworks.

A common mistake I see young developers make is thinking it’s OK to work in the “safety” of a large outsourcing company because they and a friend are working weekends on their brilliant app idea. It rarely works.

A 2012 study by Harvard Senior Lecturer Shikar Ghosh (via Silicon Valley Business Journal) found that 75% of venture-backed startups fail, although other estimates vary. Think about that for a second — if founders who are dedicating their lives to their projects with VC money fail three out of four times, what are the chances that your unfunded side project will be successful?

For those that aren’t born innovators, developers can nurture an innovation mentality by working for a company whose culture rewards “outside the box” ideas and fosters risk-taking. That can be a fast-moving corporation, a VC-funded startup, or a bootstrapped group of hungry founders.

3. Find A Great Mentor

If you don’t have a senior coder that can take you under their wing and mentor you, please go ahead and work for someone that can. This is by far the single biggest difference I see between developers who simply survive from paycheck to paycheck and those that continue to increase their opportunities until they decide to retire.

Another common mistake I see young developers make is that they think $500 per month more in the short term is more valuable than working for a good mentor in the long run. In my opinion, they are wrong — good mentors are as valuable as a top university education. 

Look for mentors who are living the life you want and who espouse the values you hold dear. Just because someone made millions and drives a Ferrari doesn’t mean they’re a good mentor for you. And a good mentor doesn’t have to be someone that makes you feel good about yourself.

Think about your best professor or teacher from your school years. Was the person you learned the most from the nicest teacher? Did they challenge you and at times make you feel uncomfortable, or did they sing your praises on a regular basis?

Mentors come in all shapes, sizes and temperaments. You want to choose those that can most accelerate your career and help you train for your “career black belt.”

As Marc Andreessen once said (paywall), “Software is eating the world.” Just make sure you get your slice of the pie while your career still has teeth. Source

Please follow and like us:
0

Top Ten industries using Python programming

April 17, 2019 Posted by Programming 0 thoughts on “Top Ten industries using Python programming”

ActiveState released a new report examining the top 10 use cases of the open source language Python. ActiveState has been building Python since 1999 and has seen customers’ use of Python grow from a scripting solution for sysadmins to web development for programmers to the driving force behind machine learning.

In a new report, ActiveState reveals ten top uses for the open source language Python. Python is an interpreted, high-level, general-purpose programming language. Created by Guido van Rossum and first released in 1991.

Use cases have grown, along with usage and adoption, as Python has evolved over the past two decades.

The top 10 most common uses of Python by industry include:

Insurance: Creating business insights with machine learning
Retail Banking: Flexible data transformation and manipulation
Aerospace: Meeting software system deadlines
Finance: Data mining to identify cross-sell opportunities
Business Services: API access to financial information
Hardware: Automating network administration
Healthcare: Predicting disease prognosis
Consulting Services: Bespoke web development
InfoTech: IT modernization
Software: Adding extensibility to legacy applications

The report is part of ActiveState’s undertaking to better understand the needs and pain points of industry related to open source languages. ActiveState aims to decrease the costs and risks of managing open source languages and enable enterprises to maximize their related benefits.

Bart Copeland, CEO, ActiveState, said: “Since our founding, we have been helping developers and enterprises eliminate costs and risks of working with open source languages. To date, millions of enterprise developers worldwide have downloaded and deployed our commercial offering of Python, ActivePython. Our goal is to help millions more. The insights reflected in this report are foundational pillars for our vision for Open Source Language Automation. Our ActiveState Platform to automatically build, certify and resolve open source languages is the linchpin to our vision. ”

Please follow and like us:
0
internal apps

5 Internal Apps That Boosted Productivity

April 16, 2019 Posted by Programming 0 thoughts on “5 Internal Apps That Boosted Productivity”

The job of a developer isn’t always restrained to the day to day minutia of maintaining a code base and taking part in stand-ups. There are times when a company may require some developers to build internal apps to increase productivity. Other times, an insightful developer may crank out an internal app to streamline clunky workflow.

Below, we’ve listed five productivity-boosting internal apps that savvy developers cooked up for their co-workers. The responses were sourced from Hacker News.

 

Face Matcher

Image result for face recognition

At a company of 10,000, it’s important to know the 100 people you’ll be working closest to. I built a “memory” game as a webapp which matched the faces of 4 people on your team to a single name and a list of self-assigned skills. You click on a photo to match a name to a face, and once you guess right a new set loads. You can randomly click through your whole team and learn a lot about them in just 15 minutes or so.

The whole thing was built with read-only SQL scripts, Flask, and some JQuery.

 

 

Shim Layer

 

I wrote a shim layer for all our packer/vagrant OS workflows to operate against an unreliable vsphere ecosystem. It exposes a suite of posix sh functions for sysadmins/developers to easily operate against this very unreliable environment. It adds automatic logging, retrying, and adjustable verbosity because of the numerous ways this environment randomly fails.

People can just . source the file in from a shared location and often find that their scripts just start to work better. It’s not perfect, nothing’s perfect. It’s not even that clever. But when builds and deploys start to work twice as good, even with the remaining failures, well, that’s something. None of the 65000 employees using it will ever know, but it feels good to know we were dropping 2/3 orders and now we’re dropping 1/3.

 

Bug Scraper

In my previous organization, we dealt with a legacy enterprise software product which had accumulated a massive bug history over multiple years and sub-products. All being tracked by an in-house bug tracking product.

Lots of issues we used to see being reported were either already fixed or had been config issues. In order to (somewhat) quickly find existing fixes/comments for issues that we get reported, I built a search tool (webapp) which scraped the bugs and comments in those bugs in order to find any relevant information around your query and listed them in order of matching probability.

Was a pretty cool learning experience to build that out. I had deployed it on a personal remote VM that devs were granted, have no idea if people are still using it.

 

 

Multithread Hack

internal apps

 

At my last job we had to do builds constantly and put it on hardware, which was annoying because builds took 15 minutes and putting it on the hardware took another 10. Couldn’t solve the latter because it wasn’t in our domain, but the first half I managed to “multithread” a build using a really hacky batch script compliation method, with a make file calling the compiler in a new command window for each file that needed to be compiled, with some checks for “needs to be compiled” or “wasn’t changed”.

An extra script at the end of the process made sure that all the compiler instances finished before continuing with the next step. All of that work got it down to 2 minutes, or in small change cases, about 30 seconds. And another part of that was integrating some configuration data with existing files, which was simple as writing up a bunch of excel macros to do the copy/pasting and file output.

It was hooked up to a shared folder on the network so the other team could just do their part, and then my part was entirely automated. In fact, the team testing things could do everything by themselves without any input from me at that point and only needed me to answer certain questions.

Yes, it’s really hacky and the whole thing is entirely silly and could have been solved by using more proper tools (i.e. not a defunct make software without wildcard support for input files or Excel for configuration), but I was VERY pleased when I got it working.

 

Bar Code Generator

 

At my current job, I saw a lab technician work manually with Excel sheets entering sample IDs and then using a website where he’d copy/paste the sample ID into to get a bar code, and then print it out to stick on the box.

I wrote a Python script that uses openpyxl module to read His Excel docs, and report lab module to generate bar codes in a PDF document with appropriate spacers so that he can simply print it out, and stick them in boxes.

He is happy and so am I that I could save his time. It only took me 20 mins to write this script.

Please follow and like us:
0
front end

Guides and Cheat Sheets for Front End Developers

April 16, 2019 Posted by Programming 0 thoughts on “Guides and Cheat Sheets for Front End Developers”

Any front end developer would love to have the brain capacity to know everything about their field. Unfortunately, we can only know so much, especially when we’re new to front end development. Google can be an excellent resource, but sometimes you want a bookmarked document that you can pull up right away to find that specific function or style attribute that will solve your problem. Or, you might be waiting for code to compile and you’re just looking for great reads that will boost your skills. If you’re anyone of these people, check out this list of front end resources that we’ve curated for you.

 

Superhero.js

If you have time to spare, you can read informative JavaScript articles written by industry heavyweights like Ady Osmani. Superhero.js serves as a repository for the top articles about “creating, maintaining, and testing a large JavaScript code base.”

Open Web CSS Reference

CSS often gets a bad rap for its unwieldiness.

However, the Open Web CSS Reference provides a categorized list of attributes that link to the W3C specification docs that teach you exactly how the attribute works.

OverAPI

OverAPI has resources for all types of developers and a few obvious useful resources for front end developers like CSS, Git, and JavaScript.

Dev Hints

The website’s founder, Rico wrote a dizzying list of front end framework and library cheat sheet that he’s shared to the public. It’s one of the most useful resources for modern front end development.

front end

The Ultimate Flexbox Cheat Sheet

Now that flex box has become more of a website design standard, it wouldn’t hurt to pour over the interactive documentation that you can find on this website.

Sass Guidelines

If you ever want to become Sass master, learning from Hugo Giraudel is not a bad way to go.

front end

Dev Docs

Dev Docs is the complete guide to everything there is to know about  languages and frameworks

 

 

Please follow and like us:
0
CSS

How Professional Programmers Use CSS

April 16, 2019 Posted by Programming 0 thoughts on “How Professional Programmers Use CSS”

Wrangling with CSS can be a nightmarish concept when you’re first learning front end web development. You’ll find that there are so many ways to accomplish a style that, if you don’t adhere to a guide, you’re page(s) will start to look like a disorganized string of naming conventions and attributes. When you’re first starting out, it can be tempting to ignore convention and just shoot from the hip to see if anything works. Rather than using classes, you might find that you have a thing for IDs, but the pros will telly you that that’s just bad practise. You can’t scale a stylesheet effectively if you litter IDs everywhere.

If you want to learn the proper way of doing things, it helps to learn from experienced front end developers. Recently, I came across an informal interview between a questioner and some devs in a forum post that I’ll share with you now. The interviewer first provided general CSS tips that will boost your skills. Some of these tips were:

  • if using grid, use height: 100vh to make the minimum height appear the height of the viewport

  • apply style using classes with semantic names instead of IDs or elements (i.e. .site_header instead of #header ul)

  • include cross-browser prefixes for any animations or exceptions (i.e. for drop-capitals in Safari or webkit- for Safari and moz- for Firefox and ms- for IE. Using an autoprefixer makes that easier.

  • convert any sizes to em and rem instead of px

  • Use grid or flexbox where available, always stick to a grid

 

I reformatted the responses so that they fit within the structure of a formal Q & A. Here’s the interview:

 

What do you do about responsiveness for mobiles when you get one layout?

 

 Ryosuke: Since mobile development tends to be one, maybe two col max, if I’m only provided a desktop version of the design, I just run with it and simplify/distill the design down. If there’s any elements that don’t work on mobile, they get cut out completely until the designer complains and sends over a more mobile-friendly design 😁 Unless you’re hired as the designer, or you’re included in the design process, it’s not your job to “get creative” or “invent” solutions to design problems that should have been resolved in the wireframe/mockup phase.

 

James Turner: Sometimes I get mobile designs, sometimes I just have to improvise. Normally though, that improvisation is pretty simple like just re-arranging items. I don’t often have to fully switch out interfaces from desktop when going down to mobile. That said, I normally do have a designer I can talk to if I really am not sure about what to do.

 

Frederick Jaime: You should always request a Desktop and Mobile layout at a minimum, if there is no way around it, do your best to adjust and make sure people are aware you’re a dev and not designer, so expectations are set. Hopefully by then, they will provide you with the additional layout(s). I don’t want to go down the rabbit hole how to design for devs because as a dev, you shouldn’t be designing.

 

Ryan: Way too project-dependent to answer. Since space is usually the concern, figure out which elements are essential and which can be relegated to a menu or removed entirely.

 

Ali Spittel: I’ve never worked on a team with a full time designer! I would agree though, that hamburgers and full screen layouts are usually the way to go.

 

Any particular choice on grid (4, 10 or 12 columns. 30em, 40em) ?

 

 Ryosuke:No preference on grid size really. I tend to use 5 and 24 cols, since they’re most prevalent in grid systems, but I’ll use 3/4/8/16/anything that looks right/makes the design work best.

 

James Turner:Honestly, I don’t actually use a grid layout. I have looked at them before but I haven’t found one that I’d want to stick with.

 

Frederick Jaime: I tend to lean into how frameworks create their grids, i normally create a sass mixin to output my columns, 12-grid fluid columns. I would suggest to look at Foundation or maybe Bootstrap. If this is a large project maybe import a framework, if this is a small project i would highly suggest to keep a list of handy mixins you can use in various project.

 

Ryan: I don’t use fixed-size grids, it’s too restrictive. I do like to have a centered content area of a fixed maximum width, since it’s hard to read extremely wide windows.

 

Ali Spittel: Definitely depends on the design, but I use fr’s a lot. Super helpful for a lot of layouts.

 

How many media queries do you include? How do you decide where the breakpoints are?

 

 Ryosuke: At minimum for any project, one media query (either for desktop or mobile), and the base CSS defaults to the other. If the design is more complex, I add media queries as necessary to correct issues at the problem breakpoint (usually at uber-wide resolutions if the site’s audience is rocking 4k double-wide monitors, or small fixes for absolutely positioned elements)

 

James Turner: I normally have quite a lot of media queries, it probably plays a bit into #2 about why. I write a new media query every time something “breaks” in the layout like if it overflowed off the screen horizontally or wraps in an unusual way. I am aiming for as fluid of a responsive layout as I can. I aim to get breakpoints down as low as 320px.

 

Frederick Jaime: Everyone should develop smallest-screen-first ( mobile-first ), and should start with two breakpoints, 768px and 1024px. Any additional breakpoints will be dictated by the design.

 

Ryan: As few as possible, obviously depends on design and that may not always be possible but I feel it’s best to have a single breakpoint around 480px. Why 480? Because your mobile layout will span 300-480 wide and your desktop breakpoint will span 480-800 or 900. You can have a layout that looks good at 480 that’s not cramped at 300, you can have a layout that looks good at 500 that doesn’t look empty at 800.

I think it’s fine to show/hide a couple of elements with more than one breakpoint too but you don’t want to be maintaining 3 or 4 complete separate layouts.

 

Ali Spittel: I normally make them once something looks weird. So once text is overlapping or something, I’ll add another mediaquery. I would also look up the sizes of popular devices and make sure your work looks great there.

Any opinions on using CSS vs JS for animations?

 

 Ryosuke:CSS/SASS for most animations, unless it’s so complex that it requires JS (JS functionality like checking animation frames or DOM elements, or using a lib like GreenSock). Going CSS-first usually ensures the highest level of browser compatibility, particularly since animations will work even when JS is disabled client-side.

 

James Turner: I use CSS to do all the actual heavy lifting of styling the animation. I will use JS to add/remove classes on a timed sequence if the animation is that complex. The browser can make better use of hardware rendering via CSS transforms etc.

 

Frederick Jaime: With animations i would suggest to use GSAP, this animation platform tries to leverage css first but defaults to js. Let’s be clear, when you refer to animations i am thinking of non-linear animations, something more then just a fade. Simple rollovers, effects should be in css but the fancy stuff should be with something similar to GSAP. You don’t want to add unnecessary weight to your page.

 

Ryan: CSS where possible. Not sure where I would have an animation that is only possible with JS. Not counting using JS to change an element’s style in order to cause a CSS animation.

 

Ali Spittel: Normally CSS animations are more efficient than JS ones, but really complex ones can only be done in JS. Transforms specifically go to the GPU which makes them super smooth.

 

At what point do you see it necessary to use SASS or LESS? Is BEM still a thing?

 

 Ryosuke:I use SASS when the project is so large I’ll be using variables across the app (which could be replaced with CSS vars) and when I need the power of mixins to simplify CSS generation (like grids, complex animations, or repetitive actions). Otherwise pure CSS run through a minification process (Gulp/Grunt/Webpack/Parcel) before it’s deployed to prod works fine. I tend to use BEM combined with OOCSS, so I have non-conflicting CSS styles (.HeaderMenu with .HeaderMenu > .list and .HeaderMenu .item nested classes). That way I get non-conflicting base styles, with without the verbose BEM sub-classes (.HeaderMenu__item) and my shorter CSS classnames work fine thanks to specificity. Similar to Airbnb’s CSS style guide (BEM component-style names), combined with Semantic UI’s (OOCSS).

 

James Turner: This is going to be an uncommon opinion but I’m not actually a fan of SASS/LESS etc. I do understand what it can provide like removing repetition from CSS and allowing for easier to manage complex styles (eg. variables). That said, I’ve never hit something with vanilla CSS that I consider a problem that needs a solution like SASS etc. As for BEM, I understand what it can help with but it is too verbose for my liking and I feel makes the styling actually more complicated. I guess these both can be summarized that SASS and BEM solve problems but they don’t solve my problems.

 

Frederick Jaime: I always use Sass, I like using a process to build my Sass. Most of the current frameworks allow you to add linters, in my current project we are using Gulp, created task to clean, compile, and minified my code.

Ryan: SCSS all the time! Here’s why a preprocessor like SCSS is indispensable for even small projects:

  • You can manage your theme colors with variables in one spot. If you decide to tweak some colors, makes updating the project super easy. Also makes sure you are staying on-brand and not using a bunch of slightly different shades.
  • You can use mixins for compatibility. Instead of having to write multiple lines of CSS for all vendor prefixes each time you use a single rule, you can write out the prefixes once and be cross-compatible with one line (this is especially handy with css grid)
@mixin select-none {
    -webkit-user-select: none;
    -moz-user-select: none;
    -ms-user-select: none;
    user-select: none;
}

span.cant-select-this {
    @include select-none();
}

Also minification and comments! It’s good practice to comment CSS, but I don’t necessarily want my comments to be made public. So a preprocessor helps with that too.

 

Ali Spittel: Bem is still a thing a lot of people like and use! I would say SCSS has the market share for preprocessers. Would go in that direction. I use it if it’s easy to set up for the project, if not, it’s not enough of a game changer for me to get it working.

 

 

Please follow and like us:
0