How to test UDP port is responding

Using the following command will do the trick.

# nc -zuv 3 192.168.123.123 5140
Connection to 192.168.123.123 5140 port [udp/*] succeeded!

Linux Biscuit!

Docker 1.8 with td-agent 2.2.1 container

This is a quick guide on setting up td-agent container with Docker 1.8

1. Create a directory to store your dockerfile and td-agent config file. 

mkdir ~/td-agent-es && cd ~/td-agent-es

2. Create a dockerfile as shown below
FROM ubuntu:14.04 ENV DEBIAN_FRONTEND noninteractive RUN apt-get update &&\ apt-get install -y --force-yes libssl1.0.0 curl RUN curl https://packages.treasuredata.com/GPG-KEY-td-agent | apt-key add - # add treasure data repository to apt RUN echo "deb [arch=amd64] http://packages.treasuredata.com/2/ubuntu/trusty/ trusty contrib" > /etc/apt/sources.list.d/treasure-data.list RUN apt-get update &&\ apt-get install -y --force-yes td-agent RUN /usr/sbin/td-agent-gem install fluent-plugin-elasticsearch ADD td-agent.conf /etc/td-agent/td-agent.conf EXPOSE 24224 CMD ["td-agent"]
3. Create td-agent.conf file as shown below
## built-in TCP input ## @see http://docs.fluentd.org/articles/in_forward <source></source> type forward # match docker logs <match docker.=""> #type stdout type elasticsearch logstash_format true host "#{ENV['ES_PORT_9200_TCP_ADDR']}" port 9200 flush_interval 5s </match>
4. Build the docker image

docker build -t td-agent-es .

5. Start elasticsearch container. As you can see from the td-agent.conf, the td-agent container will link to this elasticsearch container. Please note that this setup is not using any persistent storage mapping. You can refer to this link for more information on how to setup that mapping.

docker run -d -p 9200:9200 -p 9300:9300 --name es elasticsearch

6. Start your td-agent container 

docker run -d -p 24224:24224 --link es:es --name td-agent td-agent-es

7. Launch a hello-world to see if td-agent is forwarding log to elasticsearch container. Please note that Docker 1.8 add fluentd as one of the logging-driver. So this command will not work if you are using Docker with older version

docker run --log-driver=fluentd --log-opt fluentd-address=[your-host-IP-address]:24224 hello-world

8. Use the following command to check if elasticsearch container is receiving log. You can check container ID or container name using docker ps -a

curl -XGET 'http://[your-host-IP-address]:9200/_all/_search?q=*'

Install jobber on CentOS 6.*

All the commands below are executed using root user

1. Install go and git if you not already do so

yum install go
yum install git

2. You need to specify workspace directory, this can be any path in your system. For this example, i use the following directory as the workspace - /home/amd/go
3. Set the $GOPATH to this workspace directory

export GOPATH=/home/amd/go

4. Change directory to your workspace directory and run the following command to get the jobber files.

go get github.com/dshearer/jobber

5. As git supplied by CentOS 6 is version 1.7.1 and it cannot handle gopkg.in in package properly. If this is not down, you will encounter hang problem during compilation process. So you need to manually get the yaml.v2 package.

git clone --branch v2 https://github.com/go-yaml/yaml $GOPATH/src/gopkg.in/yaml.v2

6. Compile jobber package

make -C src/github.com/dshearer/jobber

7. Before proceed to next step, install daemonized if not already do so.

yum install daemonize

8. Install jobber

make install -C src/github.com/dshearer/jobber

9. Running the following command and you can see jobber service is running.

service jobber status

10. To create a scheduled job, you need to create .jobber file under user home directory. jobber file are written in YAML format.

 ---
- name: DailyBackup
  cmd: backup daily
  time: 0 0 13
  onError: Stop
  notifyOnError: false
  notifyOnFailure: true

11. Once it is created, you need to load the job

 /usr/local/bin/jobber reload

12. You can check the log of the job using this command.

 /usr/local/bin/jobber log

Oracle 12c ORA-04030

If you encounter such error on Windows system, please don't just look at physical memory usage but also committed memory usage which includes physical memory and page file. You could be in such situation where there is still free physical memory but running out of pagefile.

To check on the metrics, you can go to Task Manager - Performance tab - Memory. Look at the value under Committed. In this screenshot below, the system commit limit is 50.3GB and current committed usage is 9.3GB.

To find out more on what processes that consume the committed memory, you can go to Details tab. By default Commit size column is not shown in this tab. To make it visible, you need to right click on any of column title and click Select columns.

Select Commit size

You can see the Commit size column now. Among all the processes, you can see that oravssw.exe actually has much larger Commit Size compare to Memory. And this is possible the culprit for this our-of-memory issue.

So what is oravssw.exe? It is Oracle VSS Writer and it is used by a lot of backup software to perform Oracle backup. Unfortunately, there is a bug for this software that cause memory leaking according to this note (Doc ID 1358570.1)  from Oracle. There is no fix for this yet but only workaround which is to disable this service or perform service restart after backup. Although is not pretty but at least you don't need to bother with ORA-04030 issue again.