SSH to overseas server too slow?

Use China mainland cloud server as a hop.

1
ssh -v -N -L 2222:remote.oversea:2222  cloud.mainland

Then:

1
ssh -p 2222 root@localhost

Or even mount as a local folder:

1
sshfs -p 2222 root@localhost:/var/www/html ~/docker/nhweb -oauto_cache,reconnect,defer_permissions,noappledouble,negative_vncache,volname=nhweb

A few sweet ssh features to improve your development experience.

Expose local port to internet with a delegate:

1
2
3
4
5
6
7
# Remote server
sudo vi /etc/ssh/sshd_config
# Append below content:
# GatewayPorts yes

# Local
ssh -v -N -R *:9090:localhost:8080 cloud.mainland

Then heading to http://cloud.mainland:9090 for the results.

Ref:

Thanks for reading,
Regards,
Kun

Enable Google Drive file sync, on a Mac, in China

Use Polipo to convert a socks connection into a http proxy.

1
brew install polipo

Once you had polipo installed, config it to work properly with a polipo.proxy:

1
2
3
4
5
6
7
8
socksParentProxy = "localhost:8089"
socksProxyType = socks5

proxyAddress = "::0" # both IPv4 and IPv6
# allowedClients = 127.0.0.1, 192.168.1.1/255

pmmFirstSize = 16384
pmmSize = 8192

And start polipo.

1
polipo -c ./polipo.config

Then config Web Proxy (HTTP) & Secure Web Proxy (HTTPS) to 127.0.0.1:8123 (Settings -> Network -> Advanced -> Proxies -> Web Proxy (HTTP) -> OK -> Apply).

And you will get your Google Drive sync each changes on the fly.

For convenient, use our bash tool :)

Bash, OS X: pSet - a CLI, help you manage your OSX network settings.

Browse the Internet from China

Updates:

  • 2016-05-27 14:25:20

    • Docker enthusiast?

Let’s talk about network. Chinese version.

1
ssh -v -N -C -D 8089 -o ServerAliveInterval=60 -o ServerAliveCountMax=2048 rankun.org

One step further.

Before long, your will see a lot of error messages like this:

1
2
3
4
5
6
debug1: Connection to port 8089 forwarding to socks port 0 requested.
debug1: channel 24: new [dynamic-tcpip]
debug1: channel 24: free: dynamic-tcpip, nchannels 35
debug1: Connection to port 8089 forwarding to socks port 0 requested.
debug1: channel 24: new [dynamic-tcpip]
debug1: channel 24: free: dynamic-tcpip, nchannels 35

Which means your just lost the connection to remote server, but you can use autossh to monitor and restart it.

1
autossh -M 2000 -v -N -C -D 8089 -o ServerAliveInterval=60 -o ServerAliveCountMax=2048 rkus.rankun.org

3-tier forwarding

Idea: shadowsocks + port forwarding (ssh tunnel)

1
2
3
4
5
6
7
8
9
10
11
# Start a ssserver in a server outside of China (here: listen on oversea:993)
ssserver -c /path/to/config.json

# Setup a Chinese Cloud server, connect to that ssserver (rkus.json pointing to oversea:993 and listen on cloud:993)
sslocal -c rkus.json

# Finally, connect to cloud server at local (socks on local:8089, local forwarding to cloud:993)
ssh -v -C -N -L 8089:localhost:993 cloud

# If you want the local ssh port forwarding to be auto restart, try autossh
autossh -M 2000 -v -C -N -L 8089:localhost:993 -o ServerAliveInterval=60 -o ServerAliveCountMax=2048 sax.mindfine.com

The result?

Might surprise you :-)

OK, attached the video ;)

Speed of 3-tier-port-forwarding

Step closer to development:

1
2
3
4
5
6
7
8
9
10
~/docker/nhweb on  master ⌚ 1:08:21
$ git push -uf origin master
Counting objects: 5993, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (5698/5698), done.
Writing objects: 100% (5993/5993), 29.00 MiB | 5.17 MiB/s, done.
Total 5993 (delta 627), reused 1422 (delta 120)
To https://youdar@bitbucket.org/youdarnet/nanhai-wp.git
+ 1b34252...dc448a1 master -> master (forced update)
Branch master set up to track remote branch master from origin.

Push code to Bitbucket (that one, normally 10-20KB/s…) can be up to 6MB/s.

Test environment: 四川省 长城宽带…

Notes:

  • You will need 2 servers or at least ¥50 in your pocket to rent one.
  • Some Internet are bad enough, in such condition, you will need to use a AUTOSSH_POLL env to force autossh check the connection health more frequent. Let’s do a checking every 5 seconds!
    1
    export AUTOSSH_POLL=5 &&  autossh -M 2000 -v -g -C -N -L 8089:localhost:993 -o ServerAliveInterval=60 -o ServerAliveCountMax=2048 hz.youdar.net

Thank you for reading,
Regards,
Youdar

Ultimate docker on OSX collection - mapping docker files to local

Yes, you’ve heard, and have looked around here, here, or maybe here, it’s different to Linux with a simple volume mapped to your local machine. In OSX world, Docker is working in a VirtualBox…

So the file permission got be a problem, you can’t simply change its group or permission, neither. let’s fix it.

The idea is like this:

  • Everything in Docker operate as normal.
  • Create a new docker container that serves a ssh endpoint.
  • Connect to that ssh point, and get access to the file system inside our docker.
  • Get deeper, map the files of docker volume to local file system.

Let’s get started.

Run your containers as normal.

Go to your docker-compose directory, edit the file docker-compose.yml.

1
2
3
4
5
6
7
8
9
sshd:
image: 'krlmlr/debian-ssh'
ports:
- '2222:22'
environment:
- SSH_KEY=ssh-rsa AAAAxxxx_ssh_rsa_pub_content rankun203@gmail.com
volumes_from:
- data
working_dir: /var/www/html

Run it

1
docker-compose up sshd

Mount the docker file system to your local file system:

1
sshfs -p 2222 root@youdar.dev:/var/www/html ~/docker/nhweb -oauto_cache,reconnect,defer_permissions,noappledouble,negative_vncache,volname=nhweb

Notes:

  • volname=nhweb is the directory name of the mount point.
  • ~/docker/nhweb is the mount point.
  • These options are selected for more friendly and less errors.

Now your local folder ~/docker/nhweb are just the same as inside the Docker, but with the right access rights.

Notes:

  • All the file created by you will on the name of root user inside the docker.
  • If you want to use another limited user with sudo permission, use docker.

References:

Docker with wordpress

Install Docker on a Ubuntu:

Docker installation guide: https://docs.docker.com/engine/installation/linux/ubuntulinux/

If you don’t know about datastore yet, refer to: https://getcarina.com/docs/best-practices/docker-best-practices-data-stateful-applications/

  • sudo docker create -v "$PWD":/var/lib/mysql --name dbstore mysql
  • sudo docker create -v "$PWD":/var/www/html --name htmlstore php
  • sudo docker run --name mysql -e MYSQL_ROOT_PASSWORD=password --volumes-from=dbstore -dp 3306:3306 mysql
  • git clone https://github.com/youdarnet/wordpress.git
  • sudo docker run -d --name wordpress --link mysql:mysql --volumes-from=htmlstore -p 80:80 youdar/wordpress
  • sudo docker run -d --link mysql:mysql -e MYSQL_USERNAME=root --name phpmyadmin -p 3307:80 corbinu/docker-phpmyadmin

Notes:

添加新的 Wordpress 网站:

使用现有的 MySQL 数据库。

  • sudo docker create -v "$PWD":/var/www/html --name wordpressNameStore php
  • sudo docker run -d --name wordpressName --link mysql:mysql --volumes-from=htmlstore -p 800x:80 youdar/wordpress

其它提示

  • Remove dangling volumes: sudo docker volume ls -f dangling=true | awk 'FNR > 1 {print $2}' | xargs sudo docker volume rm
  • Run a nginx at current dir: sudo docker run -it --rm -v "$PWD":/usr/share/nginx/html -p 8081:80 nginx

  • sudo docker inspect --format="" nginx | json_pp

Read More

Webpage screenshot has been automatically removed!

UPDATE (2016-02-10):

I’ve found there is still a record in ~/Library/Application\ Support/Google/Chrome/Default/Preferences and deleted it, see if it works.


For those of you who suffering from the malicious Chrome plug-in called Webpage screenshot always saying it’s removed automatically…

Try this:

1
2
3
4
5
6
7
8
9
# Fire a commandline up, and go to your Chrome directory
cd ~/Library/Application\ Support/Google/Chrome

# Searching for it
find . -iname 'Webpage Screenshot'

# find out the directory and remove it, in my case:
cd Default/Extensions
rm -rf ckibcdccnfeookdmbahgiakhnjcddpki

If it does the job, then congrats, if not, post your idea below :)

Ideas come to my mind:

  • grep -r 'Webpage Screenshot' .
  • grep -r 'ckibcdccnfeookdmbahgiakhnjcddpki' .

Mongoose notes

Custom getters

Use custom getters to process the MongoDB retunred data, like an interceptor.

Schema custom getters doesn’t work with lean().

Set up the Schema:

1
2
DBSchema.set('toObject', {getters: true, virtuals: true});
DBSchema.set('toJSON', {getters: true, virtuals: true});

Add custom get function on specified field:

1
2
3
4
5
6
7
8
DBSchema.path('images').get(function (images) {
if (images && Array.isArray(images) && images.length > 0) {
images = images.map(function (image) {
return getUrl(image);
});
}
return images;
});

Read More