PAC manager: Slow clipboard paste by middle click

I have create new AWS instance and defined connection to it via PAC manager. Later I realized that pasting through middle mouse button click or pasting through context menu is incredibly slow. Almost like typing it on keyboard. Pasting through CTRL+INSERT was fast.

I suffered two days with it and did not succeed in googling that issue. Today I found out that problem was in “Wait 200 millisecs for automated char sending” setting. In rest of connections it is set to 0 so I have no idea how this setting was done – maybe I set it by incident.

pac

Gradle + Immutables + Eclipse

There is some documentation about IDE integration directly on Immutable web. Unfortunately none of proposed approaches works.

Fortunately there is Gradle plugin gradle-processors. Usage is deadly simple:

plugins {
  id 'org.inferred.processors' version '1.2.3'
}

dependencies {
  processor group: 'org.immutables', name: 'value', version: '2.2.6'
}

(Please not usage of processor instead of compile in dependencies)

That’s it! Just do gradle eclipse and restart Eclipse.

Music that resonates with my brain

I usually prefer songs with strong lyrics. But sometimes I meet melody that resonates deeply inside me. This experience is probably not universal – I doubt that wide audience would appreciate this list that high. It is more than “I like it”, it is much much deeper experience.

As you may find in post Modern approach to AI I like hierarchical model of brains. And I believe that this experience happens deep inside of this hierarchy. Recently I read William Calvin’s The Cerebral Symphony. And I get idea of multiple parallel hierarchies causing local excitement on certain level. Using darwinism are then found winners that may excite higher levels. It helped me a lot in thinking about brain structure – in previous thinking I was too limited by idea of single hierarchy and had problems with biological topology.

Resonating with music is that kind of  experience you can’t tell easily by human language – because human language was created to describe world around us – and that happens on shallow levels of hierarchy. This happens much deeper – I would not hesitate to call this deep levels soul. And people that can describe that experience are artists. Books, music, poetry, painting, films. By such methods they can describe abstract terms like love, faith, democracy, justice. And maybe build new representation deep inside of you – as sometimes happen after good book or film. And you can – if you are not an artist – express such abstract terms only using metaphors and examples. And only poorly.

Every single person having own representation deep inside may explain many clashes in society. For example Donald Trump’s internal representation of democracy is way different than mine :-)

Here goes list of my über-melodies. Do we share some of them?

Jazz:

Classic music:

World music:

Believe Driven Development

Years ago I was young programmer starting his professional programming career. I had my eyes widely opened. And I was lucky. Very first project I was assigned to was driven by boss-of-my-boss who was incredible source of inspiration.

When we get raw specification I started rolling up my sleeves and starting my IDE. I wanted to program. Now. But I was disappointed. We spend incredible amount of time on meetings with customer (it was internal one) where we tried to get not only what  customer wants, but also what is its motivation for it. And not only that – we learned step-by-step business domain of customer – vocabulary, rules, conventions, best practices, etc.

We spent so much time with it that I was more and more skeptical about finishing project before deadline. But deeper and deeper we were in discussions I more and more realized how wrong product I would build without this information.

Then there was this moment from when we started to get feeling we are discussing same topics again and again but there is no resolution to discussed problems. That was correct moment to start implementation. We were aware of many problems we may have. That was enough, it was not necessary to have answer for them. It appeared later that some of them had simple solution, some had no good solution and some were just virtual problems that disappeared. But we were aware of them.

As Dwight D. Eisenhower puts it:

Plans are nothing; planning is everything.

I don’t have to say that project was successful and that software is still in production usage (12 years now). Since then I used this approach many times and it always led to less stress development and better product when compared to other teams that uses Believe Driven Development (BDD), as I named it.

So what is BDD:

  • You believe your assumptions are correct
  • You believe you understand customer needs
  • You believe that customer understands his needs
  • You believe that you understand business domain
  • You believe that you don’t need to analyse complex environments (business rules, future requests, machines, software) in advance
  • You believe customer is providing correct information
  • Customer believe you can deliver what he dreamed of

Opposite approach of  BDD would be Assurance Driven Development (ADD):

  • You assure your assumptions are correct
  • You assure that customer knows what it needs
  • You assure you learn enough from business domain
  • You assure you understand complexity of environment
  • You assure information provided by customer is correct

Be suspicious. About customer, conventions, rules, your assumptions, … everything. Put all of this to the test. Asked for same information from many points of view. Make sure you build correct model of reality in your brain. Then start, but assure you validate your model again and again and again as time goes and new information appears. As Elon Musk does.

Log4j config: Use Tomcat context params

I wanted to use parameters defined in context.xml on Tomcat 8 in configuration of Log4j2. My context was defined like:

<?xml version="1.0" encoding="UTF-8"?>

<Context>
    <Parameter name="instanceName" value="testInstance" override="false"/>

    <Resource .../>
</Context>

Now I would like to use parameter instanceName in log filename. I spent many hours trying various approaches and googling. I wonder why is this not better documented – Using Log4j 2 in Web Applications does not really help. In the end it is easy – just use web: prefix like this:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="debug">
    <Properties>
        <Property name="log-path">${sys:catalina.base}/logs/${web:instanceName}</Property>
    </Properties>

    <Appenders>
        <RollingFile name="file" fileName="${log-path}/test.log" filePattern="${log-path}/test-%d{yyyy-MM-dd}.log">
            ...
            ...
        </RollingFile>

    ...
    ...
</Configuration>

That’s it folks!

SimpleDateFormat is NOT immutable

Yes, SimpleDateFormat is not immutable as name may suggest. This came as complete surprise to me. When using from multiple threads then appeared exceptions like:

 java.lang.NumberFormatException: For input string: ""
 at java.lang.NumberFormatException.forInputString
(NumberFormatException.java:65)
 at java.lang.Long.parseLong(Long.java:601)
 at java.lang.Long.parseLong(Long.java:631)
 at java.text.DigitList.getLong(DigitList.java:195)
 at java.text.DecimalFormat.parse(DecimalFormat.java:2051)
 at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2162)
 at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1514)
 at java.text.DateFormat.parse(DateFormat.java:364)

Quick check with documentation brought suspicion:

Date formats are not synchronized. It is recommended to create separate 
format instances for each thread. If multiple threads access a format 
concurrently, it must be synchronized externally.

Yes, it was enough to create per-thread SimpleDateFormat.

I think SimpleDateFormat is lousy name and also SimpleDateFormat breaks single responsibility principle. Much better would be decomposition into multiple classes:

  • SimpleDateFormat – immutable object only holding format of date
  • SimpleDateFormatter – mutable object used for formatting according to passed SimpleDateFormat
  • SimpleDataParser – mutable object used for parsing according to passed SimpleDateForma

REST interface polling recorder

Lately I was working on project where Server periodically polled HTTP REST interface of Agents and processed obtained data. There was only data flow from Agent to Server, so Server only reacted on Agent events and had no chance to influence Agent behavior.

schema

Because Agent was installed on special industry hardware equipment it was pretty expensive to run such Agent. Beside that laboratory test was not really similar to production usage. Writing Agent emulator from scratch would be hard from 2 reasons:

  1. Data are complex
  2. We were not sure how Agent behaves (missing spec)

So we came to conclusion we need a tool which would record REST interfaces state (JSON document in our case) during time and then replay in form of Agent emulator. Then we can record data in production and use them for development and testing. But how to store such documents into dump file? Only saving polled documents would be too disk intensive. Unfortunately there was room for optimization:

  • If document was not changed since last poll then do not save it again. During emulation is used last document until new is found in dump.
  • If document is changed just slightly (just a few fields) it may be stored if form of diff against previous state. During emulation is diff applied to previous state and new state is calculated.
  • If it makes a sense, diff or full document may be compressed. In our case using GZIP.
  • Store complete document every N minutes (same logic as key frame in movie compression field) so if file is broken or cut, emulation can be recovered.

Lines are stored into dump in following format, consisting of columns separated by pipe character:

  • timestamp
  • agent id
  • interface id
  • full x diff flag
  • compressed x uncompressed flag
  • data, stored as Base64

Currently is dump plain text file, for better compression could be of course used binary format. Using this approach we are able to record one month of Agent behavior into 50MB file!

Initially we planned to use dump solely for emulation purposes. But later on we found out many more possible use cases, like ultimate data for regression tests. For that we decided to connect data from dump directly to Server – so HTTP communication was removed. Then it was possible to test/debug but debugging week data would take a week. Hmmm.

Soon idea of working in fake real time appeared. We changed all code to not use Java functions that return real time (like new Date()) but instead TimeProvider passed by dependency injection. In production code is passed TimeProvider that returns real time, in tests/debugging is passed TimeProvider that returns current time in dump. So we can have exact test that would take a week in 1 minute. Wow.

And bonus usage: as I mentioned earlier we lacked specification of Agent behavior. But now we have dumps for many Agents in various environments lasting many weeks. We are able to analyze this data and provide to developers in clean and comprehensive form – graphs, tables, statistics.