my net house

WAHEGURU….!

Category Archives: Enlightenment

Disruption of Block-Chain in Present and Future

An immutable and distributed ledger with unique set of transactions in the forms of blocks depends on the nature and cohesiveness of each transaction which is either able to form the solution or disrupt the already existing solutions is the main event for the current existence and hype for the Block-Chain and it’s nature in the present and future.

There are certain core features of Block-Chain which includes transparency,  democracy,  Efficiency, decentralization and security, Such features can also be considered as default ones because It is the real nature of Block-Chain the provides such basic norms on the FLY!

  1. Banking: obviously!

If we talk about how and when Banking is going to be into Block-chain and how much that could bring out real change into the things then things are already there to have fun with!

Why we need Banks? – A basic institution that is able to save your money like a wallet and you are able to send it all over the world. Right now just believe that we are not talking about Investments as well.

Download ABRA and just Enjoy the Bank in an APP: https://www.abra.com/

Barclyas is the bank that uses the BlockChain:

15% of Banks will be using Block-Chain by the end of 2017:

2. Cyber Security:

Data is stored in the encrypted form and each time it is verified by multiple parties while storing it in DB so that means each time Data is able to secure by default. That eliminates the need of Third party security systems!

3.  Supply-Chain Management! (OMG that was DOPE when I heard that how Efficient it could be in Supply-Chain management):

All the transactions are Documented in permanent record and are being watched securly when any assest is moved from one party to another in the supply chain management!  That is able to measure time, Cost, labor and Emissions which is helpful to reduce and form the necesary  plan to make complete Efficient Supply-Chian management! Which is also helpful to verifity the authenticity and fair trade status of the product.

Startups that are working on Supply-Chain Management are as follows:

https://www.provenance.org/whitepaper

https://bravenewcoin.com/news/fluent-unveils-blockchain-based-global-supply-chain-software-platform/

http://www.skuchain.com/

http://www.blockverify.io/

4. Forecasting:

Think of world like where each and every person is able to collaborate their brian-power and able to build and things collaboratly either it is a research work or making a staretgy that is able to predict next stock price in the NASDAQ!

A decentralized prediction market solution: http://www.augur.net/

5. Networking and IOT:

Thinks of different nodes and how these are able to communicate with each other that could bring repairing and updating and repairing itself without even need of external human maintainer! All the nodes would be able to connecet to each other with eliminating the need of centralized system.

One can see whole idea about ADEPT here: https://www.slideshare.net/_hd/ibm-adept

6. Insurance:

Insurance is based on trust management. BlocakChain can be used to verify many types of data in insurance contracts! Like the real identity of insured person as well as his/her medical records as well.

One example of such system is Aeternity: https://aeternity.com/

7.  Private Transport and Ride Sharing:

There are various cases of Block-Chain which can  be helpful for peer to peer ride sharing system. Presently most of the Ride sharing or Taxi booking systems are based on the some kind of Centralized system like Uber and OLA but using Peer to Peer Transactions and communication can be improved and cost of middle man or centralized system can be reduced over a large amount!

Great Examples are Arcade City and Lazooz:

https://cointelegraph.com/news/arcade-city-decentralized-blockchain-based-answer-to-uber

http://lazooz.org/

8.   Cloud Storage:

It was always a challenge  only because we always need to secure data and have it in many different places and those places should be on virtual locations not in one but also in many!

Storaj is one example of that: https://storj.io/

9. Charity:

There are many cases of corruption comes into place when there are money collection and resources utilization in the charity, There is no doubt that in the cases of Charity because of less transparency all the Money of Charity could be involved into wrong things as well. BlcokChain could be a very useful to get to know that where the donations are going!

https://www.bitgivefoundation.org/

 

10.  Voting:

Block-chain can also be used for Registration, Verification and Vote-counting. Creating and immutable, Publicly viewable ledger of records is the real way to make elections more democratic and viewable.

https://www.democracy.earth/

https://followmyvote.com/

11. Government:  Most of the government based systems are slow, in-efficient and biased. To slow down the bureaucracy,increase security, efficiency and to make government systems more transparent BlockChain is so far the best solution as well.

https://www.khaleejtimes.com/editorials-columns/blockchain-future-is-here-and-dubai-has-grabbed-it

12. Public Benefits:

Public Benefits is the one of the system that has suffered each time from slowness and bureaucracy, As one of the major thing of being transparent block-chain here is the great player to implement that all benefits towards public can be shared in transparent way.

Govcoin is one of the example of BlockChain that is disrupting welfare  system in the UK:

https://www.blockchaintechnology-news.com/2017/11/28/govcoin-aims-disrupt-uk-welfare-system/

There is one more project which is called circle that is able to implement Universal Basic Income using BlockChain: https://www.circle.com/in/

13: Health-Care:

Hospitals always  lack of need to save secure data and that can be used foe various treatment in the patients. BlockChain can be used to save and secure Medical data of patients and that can be shared with Authorized doctors and patients.

Following are the two startups working on the BlockChain to work on secure data:

https://gem.co/

https://tierion.com

14.  Energy Management: Energy Producers and users  have to go through a long list of middle grids to get resources from each other.

Transactive Grid is blockchain solution that is able to create De-Centralized way share resources between various energy producers and consumers.  Transactive Grid uses the Ethereum Blockchain.

15. Online Music:

There are several Block-Chain solutions those are working on the problem for selling music from producers to users. Smart contracts can also be used directly to solve the issue of licensing.

http://myceliaformusic.org/

https://ujomusic.com/

16. Retail:

Shopping is always based on trust on the shop or the market place but in real there is no particular way to understand that how good or bad product could be inside the wrapper. BlockChain is able to cut all the middle-man between buyers and sellers so each time time. Smart contracts are there to maintain the credibility of the system.

https://www.openbazaar.org/

https://ob1.io/

17. Real State: There are few issues in real estate like Fraud, Lack of transparency and  mistaken in public records. BlockChain can simply reduce the paper based record keeping structure. Block-Chain implementation can simply help with tracking, verifying ownership, ensuring accuracy of documents and transferring the deeds.

The one company that is solving this problem is https://www.ubitquity.io/web/index.html

18. Crowd-Funding:

In Blcok-Chain Crowd-funding is created through smart-contarcts and online reputation system. New products can raise funds by releasing their own tokens and those tokens can be used later to buy products and other services.

19 OUR industry: Telecommunication and Networking. SDN(Software Defined Networking!)

Securechain delivers security, scalability and auditability to Software Defined Networks.

SYTEL REPLY HAS DEVELOPED A WORKING PROOF-OF-CONCEPT TO SHOW THE POTENTIAL OF THIS SYSTEM, BASED ON THE ETHEREUM BLOCKCHAIN (THE SAME BLOCKCHAIN AS CHOSEN BY THE R3 FINANCIAL CONSORTIUM, MICROSOFT
AND IBM/SAMSUNG), WHICH CAN NOT ONLY ALLOW VALID NETWORK ENTITIES SUCH AS SWITCHES TO ENTER THE NETWORK, BUT TO DETECT AND REJECT HACKING ATTEMPTS.
Paper based on Block-Chain and SDN:

The way of superior man! Small Digestion!

There are three stages in the life of a man!

Satge1: He thinks for himself only.

Stage2: He thinks for his Family, world and whole earth.

Stage3: He has his true purpose to live and he lives as Gift for the whole world, He has no worry to please anyone and and he has no worry as well to make anyone happy, he lives the way he wants to live and all the time whatever happens in his life and world around him gets huge potential, love, care and benefits for him! that is the way of the superior man! He is the gift for the whole world.

Very bold chapters in the book:

  1. Stop Hoping For a Completion of Anything in Life.(work is never ending Process)

2. Live As If Your Father Were Dead.

3. Be Willing to Change Everything In Your Life;

4. Women Are Not Liars;

5. You Will Often Want More Than One Woman;

6. Your Excellent Track Record is Meaningless to Her;

7. She Want’s the “Killer” In You;

 

Quote1: Keep going that is just life and the way.  LIfe is a journey not a destination!

“Most men make the error of thinking that one day it will be done. They think, ‘If I can work enough, then one day I could rest.’ Or, ‘One day my woman will understand something and then she will stop complaining.’ Or, ‘I’m only doing this now so that one day I can do what I really want with my life.’ The masculine error is to think that eventually things will be different in some fundamental way. They won’t. It never ends. As long as life continues, the creative challenge is to tussle, play, and make love with the present moment while giving your unique gift.”

Quote2: Are you really living up-to full potential that you don’t care if you just die! Fear of death goes away when you take life up-to full potential.

“If you were to die right now, what would be the feeling texture of your last moment? Are you feeling the infinite mystery of existence, so that your last moment would be one of awe and gratitude? Is your heart so wide open that your last moment would dissolve in perfect love? Or, are you so absorbed in some task that you would hardly notice death upon you, until the last instant, whoosh, and everything is gone?”

 

Quote3:  Beat mediocrity do something that you have done before! Embrace your limits and get crazy enough sometimes if possible.

To help you remember the triviality of your daily tasks, interrupt your schedule with refreshers. These refreshers should cut to your core and strip the fat off the moment. Consider your own death. Behold an image of the most enlightened being you know. Contemplate the mystery of existence. Relax into the deepest and most profound loving of which you are capable. In your own way, remember the infinite, and then return to the task at hand. This way, you will never lose perspective and begin to think that life is a matter of tasks. You are not a drone. You are the unbounded mystery of love. Be so, without forgetting your tasks.”

 

Quote4: Self discipline is the key to success.

The core of your life is your purpose. Everything in your life, from your diet to your career, must be aligned with your purpose if you are to act with coherence and integrity in the world. If you know your purpose, your deepest desire, then the secret of success is to discipline your life so that you support your deepest purpose and minimize distractions and detours.”

Quote5: Give it a little more than you can do. A little progress everyday!

In any given moment, a man’s growth is optimized if he leans just beyond his edge, his capacity, his fear. He should not be too lazy, happily stagnating in the zone of security and comfort. Nor should he push far beyond his edge, stressing himself unnecessarily, unable to metabolize his experience. He should lean just slightly beyond the edge of fear and discomfort. Constantly. In everything he does.

Quote6: Face your fear and embrace chances of success.

“By leaning just beyond your fear, you challenge your limits compassionately, without trying to escape the feeling of fear itself. You step beyond the solid ground of security with an open heart. You stand in the space of unknowingness, raw and awake. Here, the gravity of deep being will attend you to the only place where fear is obsolete: the eternal free fall of home. Where you always are. Own your fear, and lean just beyond it. In every aspect of your life. Starting now.

Quote7: Be the best version of yourself! the one from the one who was yesterday!

Make your life an ongoing process of being who you are, at your deepest, most easeful levels of being. Everything other than this process is secondary. Your job, your children, your wife, your money, your artistic creations, your pleasures – they are all superficial and empty, if they are not floating in the deep sea of your conscious loving.”

Quote8:  There is no shame in accepting your fears because these are just fears nothing else after all.

“A free man is free to acknowledge his fears, without hiding them, or hiding from them. Live with your lips pressed against your fears, kissing your fears, neither pulling back nor aggressively violating them.”

Quote9: Whatever you are and whatever is happening to you, you are the only one who is responsible for that!

“You are entirely responsible for cutting through your own laziness, addictions, and unclarity. There is nothing to wait for and nobody to blame. Whatever techniques are appropriate, use them. Try talking with your friends, using therapy, practicing meditation or prayer, going on a vision quest, reading scripture, walking in nature, keeping a journal, or studying with a teacher. Remember that your success with any method you choose depends entirely on your actual commitment to discovering your deepest truth and aligning your life with it.”

Quote10: The best you can get from anyone is yourself and yourself only! It is the way.

“Once you have grown into independent adulthood, you no longer need somebody to take care of you. You can be responsible for yourself. In particular, you realize that you are responsible for your own happiness. Nobody can live your life for you. You must create your own health, success, and happiness. This sense of self-responsibility is only a partial maturity, however. Beyond self-responsibility lies the responsibility to give your gift. It is important to grow beyond dependence on your intimate partner for your own happiness. But it’s equally important to grow beyond simple independence and autonomy. The next stage of intimacy after personal independence has been attained is the mutual flow of gifting, or serving each other in love.”

Laws of Power(48 or more may be!)

Power:- Hmm, what it is and how we define power and how we are able to acquire and what are those things behind it’s existence those make humans more crazy about it. Knowledge is Power? Or Money is Power Or self-Awareness is power? Physical strength and Muscular body are also great examples of Power But what is real scenario behind power that lead Humans towards total destruction or behind great achievement.

images (1)

There is interesting saying of Robert Kiyosaki that explain many things about Power when you are young and how you can attain it or how you might be attaining it:

48 Laws of power could be Evil if you are not able to understand the real meaning behind the book. Now here are some practical aspects of book those many of readers might find attractive or really evil if someone is hungry for power and someone is desperate to learn how to acquire power.

1. Never Outshine the master:
Those who are above you always make them Great and Prosperous because you have to learn so much stuff from them.

2. Never Put Too much Trust in Friends, Learn to use Enemies!
In real way yo are always alone and you need to understand that loneliness is solitude and that solitude is the key to overcome your mental abilities. Most of the times when someone is too much surrounded by friends/Yes-Mans He or She gets too comfortable but on the other hand uncomfortable Environment with Enemies lead you to grow more!

3. Stay Mysterious as much as you can, There must be only one person in the whole world who must be knowing your true goal/mark and that person is you and only you! You must remember only one thing that you are at war and only you must be knowing your Next Move!

4. Speak Less but speak Effective whenever you are going to speak something.  Say less that means you would be able to get caught less about stuff that you are going to say about yourself.

5. Your reputation is the only thing that can help you to get unlimited power even when you are not working on it, It is so powerful law but you have to invest small chunks of time everyday to work on this law.

6. Don’t be attention seeker but learn to get attention whenever you are required to have it.

7. You must know or learn to know how Others will work for you and how you must make others realise that they would be doing great not for you but for themselves. All the time make people believe that they are competing against you or they might be doing things those might effect you because the people who are working for you are your enemies and that those enemies of yours will work so much for you only because they want to feel/make you low but in reality you know when you have to cut them and inherit all the stuff they did and make your wrapper around whole work your enemies have done for you. 🙂

8. If you need to force someone to come to you then you are already lost.:)
The force that you will apply on someone else will only make you week and look desperate and that desperation of yours could make someone scary. You never need to force someone all your energy should be around inventing the bait and that bait is the real way for someone to fall for you.

9. Arguments will only make your mind into less peace and more towards the complications and envy. Your arguments will make things more into deep where hope to make stuff happen move towards in less space. Although you might be cool and go towards your work and actions those could prove one to do things which are not on the real scenario but at end you will feel that when others were busy at arguments you took the lead and made the things happen. 🙂

10. Reality is that if there are positivism and affection resides inside the world there are also Negative thoughts and great amount of un-luckyness. Run from such people as much as you can and never look at them at all. If you will find the people in your life those are pulling you backward then you will never get ahead. Such people could also be your relatives or Loved-ones.

11. The more you will teach someone more he or she will go towards independence and more they will come to attack you back or will go so far away from you that you would never be able to get required benefits from them, Such cases are so common that most of the BIG industries follow those rules, Campus Placements are one example of such Law of Power. 😉

12. Learn to know where you have to apply honesty and how much it would effect your circumstances, Be nice to people when you have to be because then you would be able to make things more into realistic for your work, More nice and honest you will be towards other people for certain situations more you will be able to get their side for yourself.

13. Lure them for more and more things in those are for their self interest and they will only come to you when they will find that they are attracted towards you.

14. You are always nice to everyone and all the time you need to learn and understand how someone is behaving with you, in each case you must be going through many levels of layers when you are going to reveal about someone.

15. Wounded lion is far more dangerous than Healthy one, If you got your enemies into the world where they have lost from you once or twice than all you need to do kill them, Crush them properly because at any case if you will leave them un-touched you will regret at the END, Very MUCH!

16. Too much Availability is curse and If they get used to you they will never realise your true Value and that make them think and feel less about you, so use your presence only when it is effective most.

17. Stay Mysterious because that is the Ultimate gift you can give yourself. All the time they should be in-confusion that what would be your next move or how you are going to perform in next 3-4 days or what next you could say about stuff and things. Air of Mystery about you is the way they will never be able to understand your true Potential and your true findings. Less predictable you are less vulnerable you will be.

18. Sometimes just stay in the group and you will find that you have very good amount of Surroundings and security that will make you not shining like a Gold, That make sense because shining and shining alone all the time could cause huge amount of attraction towards you and you would be the one who anyone could be able to make a hit, Prevent yourself from attacks. Never Get ISOlated.

19. You must be knowing whom you are attacking before you are going to attack, Someone out there with really normal outlook could be super dangerous from outside, If you are mysterious anyone else could be as well, So learn to know that whom you are going to attack and how much loss that would cause you, So each time whenever you meet a mysterious person either stay away from that or be-aware very much before calling him out.

 

20. Your Real commitment should be only towards yourself and yourself only, your commitment to others could be more dangerous for both parties, one is your too much availability and other is how your availability towards others could decrease your time and at last but not least is your commitment to one side could also lead you to be enemy of OTHER-SIDE! either way you should avoid creating enemies of yours.

21. You are fool one ever anyone has seen, That should make them to think about you Because only then you would be able to know that what actually the other person is and how he/she is attack-able. 😉

22. Sometimes you have to tell them or make them believe strongly that you are lost and you are going to surrender yourself now, Only then you would be able to re-gain power and make yourself be more consistent. Re-gain your power and you will be able to attack again and this time you would be able to attack again and this time they will believe that only you would be able to loyal to them but actually you are having greater plans this time. 🙂

23. There is so much noise all the time around you, Most of the time this noise is major cause for you to loose focus on real things and waste HUGE amount of Energy, Time and money on such things could be great productivity killer for you, So each time you need to have focus on most important task and energy thing.

24. You must be the one who is the best if anyone asks something for advise but you are or should be only good to advice to those who are have the Designation of King.

25. Power of Re-Invention and Revolt is the most interesting power you can have for yourself, Each time society and surroundings tries to box you up into certain kind of Image and someone at some point you accept that live that kind of boaring life, All you need to do is always get out of the box and re-Invent yourself because It’s your game, your life and you must be the only one who should be controlling it!

26. Never get caught, Learn to make a good amount of balance and you would get to know that you can grow anywhere and you can move everywhere. You do the bad things but you should never be the one who actually performed such things! 😉

27. Be the one whom others feel that is the one that is desired most, Nobody takes care of your causes and things, All you need to do is make them realise that you are helping them not they are helping you to reach your goal.

28. Your Boldness is the key to make believe that whatever you are going to perform had great potential and you believe in that action. That is the only way you are going to rock each time, There is one more catch in this situation and that is if you are un-sure about something that you feel could be not that good to perform then you are good to go along with someone else who has much more idea of what we are about to do!

29. End goal is the most important you must be knowing and that END-GOal is everything, Because you need to fight and along the fight you need to know where you have to stop.

30. Be Cool! No be just Super-COOL! if they would be knowing that you are only doing stuff that is making you week the you are already vulnerable.

31. You are the one who must be controlling the main point or the centre source of Power and let others Play with small things and believe that they are the one who are actually controlling the situation.

32. Never be the one who is delivering bad News and in this whole point you must be the one who is not delivering TRUE BAD news, Learn to please people and learn to make them the one who want to listen only great news.

33. Learn to find out other’s Weakness and learn to Exploit them on that particular Weakness and moreover learn to Penetrate into other’s mind, There could be many weaknesses of someone and those could be Desperation,Fear, Loneliness, Charm, Drugs and Knowledge.

34. You are Amazing, Awesome, Att, Ghaint, Siraa and you are the one who is ruling the World, Live life king Size, You are the ALpha and you must Un-leash this thing in your life because that is what you are made to do. You are Lone wolf, Wolf on the top of the Hill is Never as Hungry as Wolf on climbing the hill.

35. Stay Calm Stay Cool, Work is Never Ending Process, Fight is Never Ending Process, Each day is War and you are not fighting to Survive now it’s your Attitude to Survive so that has very simple meaning and that simple meaning is You are cool enough. Your Hurryness is your Best Exploiting point and you can always get Exploited by someone at any level.

36. There is an art and that is: “Subtle art of Not Giving a Fuck!!” You must not give any kind of attention to your enemies and that less-attention makes them Crazy for you.

37. If you have to explain yourself your power is already in question. Your search for power depends on shortcuts. You must always circumvent people’s suspicions, their perverse desire to resist your will. Images are an extremely effective shortcut: Bypassing the head, the seat of doubt and resistance, they aim straight for the heart. Overwhelming the eyes, they create powerful associations, bringing people together and stirring their emotions. With the white light of the moon in their eyes, your targets are blinded to the deceptions you practice

38. Never Fight the Environment and all time just do your work under the hood, Never complain against the Crazy Mob, just do the stuff that you think+believe is great!

39 Make them know that they are the only one who will loose and that makes them desperate and Crazy to go against you, and the craziness will cause huge amount of imbalance for them, When they are angry and imbalanced that is your time to Rock!!

40. If it is free It is Dangerous.

41. Live away the Shadow of Great People! No matter how great you are doing or How amazing you could be, Each time you just have to remember that You need to make your own Existence. And that Existence of yours helps to create you your own things, Your Own world and your Own Treasure.

42. Do not wait for the troubles they cause to multiply, do not try to negotiate with them – they are irredeemable. Neutralize their influence by isolating or banishing them. Strike at the source of the trouble and the sheep will scatter.

43. Understanding that one alienated, disaffected soul can spark a blaze of discontent, you too must constantly win over more allies on all levels—a time will inevitably come when you will need them.

44. Do as what they do, That will make them to that you are not against there will or may be you are just doing what they are thinking to do, That will only make them crazy enough to fall for you ar make so much crazy and Entitled.

45. Turn the tables but do it slowly, At the end you know that real power is to run your own show, and to run your own show you don’t need to do things as fast as possible, So to have that possibility you need to act fine and accordingly.

46. You are not perfect and never appear the one, Too much shine cause too much attraction and that much attraction cause too much Evils of others.

47. Excess of Everything is Bad and that bad you need to identify. You need to know that when you have won and Do push things sometimes below the required limits.

48. Be Water My Friend! Be water. Flow like a water, Act like a water, Roll like a water. Stability is an illusion.

wooooo hooooo!!

images (2)

TPOT Python Example to Build Pipeline for AAPL

This is  just first Quick and Fast Post.

TPOT Research  Paper: https://arxiv.org/pdf/1702.01780.pdf


import datetime
import numpy as np
import pandas as pd
import sklearn
from pandas_datareader import data as read_data
from tpot import TPOTClassifier
from sklearn.model_selection import train_test_split

apple_data = read_data.get_data_yahoo("AAPL")
df = pd.DataFrame(index=apple_data.index)
df['price']=apple_data.Open
df['daily_returns']=df['price'].pct_change().fillna(0.0001)
df['multiple_day_returns'] =  df['price'].pct_change(3)
df['rolling_mean'] = df['daily_returns'].rolling(window = 4,center=False).mean()

df['time_lagged'] = df['price']-df['price'].shift(-2)

df['direction'] = np.sign(df['daily_returns'])
Y = df['direction']
X=df[['price','daily_returns','multiple_day_returns','rolling_mean']].fillna(0.0001)

X_train, X_test, y_train, y_test = train_test_split(X,Y,train_size=0.75, test_size=0.25)

tpot = TPOTClassifier(generations=50, population_size=50, verbosity=2)
tpot.fit(X_train, y_train)
print(tpot.score(X_test, y_test))
tpot.export('tpot_aapl_pipeline.py')

The Python file It returned: Which is real Code one can use to Create Trading Strategy. TPOT helped to Selected Algorithms and Value of It’s features. right now we have only provided ‘price’,’daily_returns’,’multiple_day_returns’,’rolling_mean’ to predict Target. One can use multiple features and implement as per the requirement.


import numpy as np
import pandas as pd
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.model_selection import train_test_split

# NOTE: Make sure that the class is labeled 'target' in the data file
tpot_data = pd.read_csv('PATH/TO/DATA/FILE', sep='COLUMN_SEPARATOR', dtype=np.float64)
features = tpot_data.drop('target', axis=1).values
training_features, testing_features, training_target, testing_target = \
            train_test_split(features, tpot_data['target'].values, random_state=42)

# Score on the training set was:1.0
exported_pipeline = GradientBoostingClassifier(learning_rate=0.5, max_depth=7, max_features=0.7500000000000001, min_samples_leaf=11, min_samples_split=12, n_estimators=100, subsample=0.7500000000000001)

exported_pipeline.fit(training_features, training_target)
results = exported_pipeline.predict(testing_features)

Socket Programming and have fun with Python

Client Socket and Server Socket:

Client Computer like your browser or any piece of code you want to talk to your server uses client socket and Server uses both client and server socket.

Sockets are great for Cross-Platform communication.

Following is minimal Example of Socket and stuff:

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("www.python.org", 80))

What is INET?What is Sock_Stream?

Almost that is all happened on client side, When connect is completed socket that is ‘s’ we just created can be used to send and request the specific text page requested. This socket will be read and reply, after that it will be destroyed. Client sockets are normally only used for one exchange (or a small set of sequential exchanges).

Now let’s look what is happening at server side:

# create an INET, STREAMing socket
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# bind the socket to a public host, and a well-known port
serversocket.bind((socket.gethostname(), 80))
# become a server socket
serversocket.listen(5)

 

Generators, Context-Managers and Coroutines:(Completing course)

Generator Pipelines:

  1. Several Pipelines can be linked together.
  2. Items flows one by one through the entire Pipeline.
  3. Pipeline Functionality can be packaged into callable functions.

One Example of Generator pipeline:

def seprate_names():

for full_name in names:

for name in full_name.split(‘ ‘):

yield name

full_names = name.strip() for name in open(‘names.txt’)

names = seprate_names(full_names)

lengths = ((name,len(name))for name in names)

longest  = max(lengths,key=lambda x:x[1])

Another approach is as follows is one wants to use using Function name: anotherapproac

 

Context-Manager:

Why we need context-Manager?

‘with’ statement in Python that we use to do file operations is Context-manager. It is something like to “Have the state and open that state and with-in that state to do things”. using with in Python we open file and till the file is open we do some good things and after doing all good things we close the file. So ‘with’ is a context Manager using that we make the state of file open and after that we do all the things we need to do with file.

Other useful cases of Context-manager:

Close/Open File/Socket(Even it crashes)

Commit/Fetch (Even if crashes)

release the lock (Even it crashes)

When you really need Context-Managers?

At last fun not least, If I will be creating a Chat-BOT in Python then I would be able to use Context-Manager in Python so I would be doing some stuff and after completing that stuff I can go out and have fun.

when_need_context?

So we use @staticmethod in Python class,  that means no matter what happens we will be able to run this method at any cost.

That is just using decorator. Now if we want to create a context manager using decorator?

So what is Context-manager?

  1. I have to go to a particular directory, list all files with .txt extension, then come-back to current location (Simple use case)

context

  1. I have to load specific ML model, I have to predict against several parameters and get results, Return at specific state(un-load the model)
  2. I have to open Socket connection, Read various kind of data, close Socket connection.

On the other way we can also write it like:

Setup

Yield-control

Wrap-UP

Context-managers are powerful tool to make code more modular and Succinct.

Now What if we have to use Context-manager as Yielded value?

interesting_context

May be little-bit  more about COntext-managers-

https://jeffknupp.com/blog/2016/03/07/python-with-context-managers/

What are Coroutines and how we have to handle those?

  1. Receive Values
  2. May not return anything
  3. Not for iteration

What is the design of Co-routine:

  1. Receive Input
  2. Process that Input
  3. Stop at yield statement

send() method is used to send value to coroutines.

 

More uses of Co-routines:

Coroutines are really powerful for Data-Processing Operations.

https://stackoverflow.com/questions/303760/what-are-use-cases-for-a-coroutine(Concurrency and Multi-Processing)

One of most important course about Co-Routines/Concurrency and really interesting way to handle multiprocessing: http://www.dabeaz.com/coroutines/ —think win win—-

 

Common Regular Expressions

Find line in multiple files:


grep -rnw '/home/ubuntu/workspace/tools-tpn-ops/' -e 'import os'

case insensitive: grep -i "Aditi" helloaditi

case ins and separate existance: grep -iw "is" helloaditi

match and show three lines after that: grep -A 3 -i "am" helloaditi

case of exclusion: grep -v -e "arsh" -e "honey" helloaditi

find position of word in file: grep -o -b "aditi" helloaditi

find line number of matching text: grep -n "sharma" helloaditi

read a specified line number: if you exactly know the line number then do

lets say you want to read line 84
head -84 a.txt| tail -1

Engineer’s Guide to Julia Programming

Engineer’s Guide to Julia Programming

Finally the moment has come when I can say that I can be productive as well as my solution can be Parallel,Optimize-able,Customizable and at last but not least glue-able. Yes those are the fantastic features I believe one can rely on while Learning any New Programming language and Developing a Very High Quality AI/ML Embedded Software Solution.

Why?

Julia Solves Two Language Problem.

Important Disclaimer for Newbies: I am Pythonista by choice and over the last few years I have Developed Projects using Python and it’s sister technologies to provide the solutions those are related to

Automation(Python -Scripting)

Web-Development(Django,Flask,Sanic,Tornado)

Data Analysis(SageMath,Sympy,Paraview,Spread-Sheets,Matplotlib,Numpy,Scipy,SKLearn)

Quantitative-Analysis(Quantopian.com)

3D Modeling(FreeCad, BIM,IFC), and Cluster Computing(Rock’s Cluster).

Now I just wanted a tool that would allow me to write Pure-mathematical expressions(using required signs not variable names) and write Machine-Learning/Artificial-Intelligence/Deep-Learning code where I would find myself on core layer of abstraction not like Tensor-Flow, Pytorch, or Numpy/Pandas. I am not against these libraries those helped me “soooo” much over the years but I have no idea that what kind of things are happening under the hood and may be I will never be allowed to change the working internals of Numpy/Pandas/Cython or anything that is related to Scientific Python only because there could be large amount of Fortran/C++ or Pascal kind things and crunching numbers as well.

Stuff that an Engineer need to perform for various kinds of jobs in Julia-Programming Language can be described as follows:

Solving a Simple Mathematical Equation in Julia:

A = randn(4,4)
x = rand(4)
b = A*x
x̂ = A\b # here we have written x-hat Symbol
println(A)
println(x)
println(x̂)
@show norm(A*x̂ - b)

Doing Matirx Operations in Julia

A = randn(4,4) |> w -> w + w' # pipe A through map A -> A + A' to symmetrize it
println(A)
λ = eigmax(A); # have you checked the lambda?
@show det(A - λ*I)

Performing Integration:

Performing integration might be one of the most important task one would be doing in Day to day if someone is involved with problems related to modeling and designing a solution using CAS(Compute Algebraic System) like Matlab or Sage-Math but designing a solution using CAS and then finding various ways to implement it into production is kind of “LOt of WoRk” I assume that only come with Either Experience or Lots of Extra Brain cells. 😉 See here Julia Plays an important role: “Solving two Language Problem”.

# Integrating Lorenz equations
using ODE
using PyPlot
# define Lorenz equations
function f(t, x)
σ = 10
β = 8/3
ρ = 28
[σ*(x[2]-x[1]); x[1]*(ρ-x[3]); x[1]*x[2] - β*x[3]]
end

# run f once
f(0, [0; 0; 0])

# integrate
t = 0:0.01:20.0
x₀ = [0.1; 0.0; 0.0]
t,x = ode45(f, x₀, t)

x = hcat(x…)’ # rearrange storage of x

# Side-Note::: What … is doing in Julia? (Remember *args and **kwargs in Python?)
# for more see:
goo.gl/mTmeR7

# plot
plot3D(x[:,1], x[:,2], x[:,3], “b-”)
xlabel(“x”)
ylabel(“y”)
zlabel(“z”)
xlim(-25,25)
ylim(-25,25)
zlim(0,60);

Really interesting Dynamic Type System()::

This is one of the most interesting part for me to have so much fun with Julia and it’s GREAT! Type System, You know why? Because It knows how long that bone is and how much Calcium will be there:

### Built-in numeric types

Julia’s built-in numeric types include a wide range of
1. integers: Int16, Int32, Int64 (and unsigned ints), and arbitrary-precision BigInts
2. floating-points: Float16, Float32, Float64, and arbitrary-precision BigFloats
3. rationals using the integer types
4. complex numbers formed from above
5. vectors, matrices, linear algebra on above

Ok let’s Have The Fun!

I encourage you to run following code into Jupyter Notebook that is running With Julia-Kernel.

π

typeof(π)# it will return irrational. Beacuse pi is irrational Number? 😉

Let’ Hack Julia’s Type System on Much deeper level!(Because it is much more than classes)

What else we need to know about it?

Define new Parametric Type in Julia:

type vector_3d{T<:Integer}
x::T
,y::T
end

# can we call x any as Data-Members as like as C++ Data-Members?

type_call = vector_3d{25,25} # this is how we call it.

Let’s Just make Types more interesting: (and immutable)

immutable GF{P,T<:Integer} <: Number
data::T
function GF(x::Integer)
return new(mod(x, P))
end
end

Deep Learning and Machine Learning in Julia:

In the real eye Julia is developed to write “Mathematical Functions” by just using Native Language Syntax. It is more like if you want to do linear regression rather than installing a New_library and calling inbuilt Linear function of that library those could be written in C, C++ or Fortran may be or More or less Optimized Cython-Python Magic. But Julia responsibly provides static inbuilt and Really fast code methods to write your Own linear regression as easy as Python and as Fast as C++/Fortran.

Available Machine-Larning Packages in Julia:

Scikit-Learn in Julia:

ScikitLearn.jl implements the popular scikit-learn interface and algorithms in Julia. It supports both models from the Julia ecosystem and those of the scikit-learn library (via PyCall.jl).

https://github.com/cstjean/ScikitLearn.jl

Text Analysis in Julia:

The basic unit of text analysis is a document. The TextAnalysis package allows one to work with documents stored in a variety of formats:

  • FileDocument: A document represented using a plain text file on disk
  • StringDocument: A document represented using a UTF8 String stored in RAM
  • TokenDocument: A document represented as a sequence of UTF8 tokens
  • NGramDocument: A document represented as a bag of n-grams, which are UTF8 n-grams that map to counts

https://github.com/JuliaText/TextAnalysis.jl

Machine-Learning Package with name Machine_learning:

The MachineLearning package represents the very beginnings of an attempt to consolidate common machine learning algorithms written in pure Julia and presenting a consistent API. Initially, the package will be targeted towards the machine learning practitioner, working with a dataset that fits in memory on a single machine. Longer term, I hope this will both target much larger datasets and be valuable for state of the art machine learning research as well.

https://github.com/benhamner/MachineLearning.jl

Deep Learning in Julia:

Mocha is a Deep Learning framework for Julia, inspired by the C++ framework Caffe. Efficient implementations of general stochastic gradient solvers and common layers in Mocha can be used to train deep / shallow (convolutional) neural networks, with (optional) unsupervised pre-training via (stacked) auto-encoders. Some highlights:

https://github.com/pluskid/Mocha.jl

Deep Learning with Automatic Differentiation:(What is automatic Differentiation?)

Knet (pronounced “kay-net”) is the Koç University deep learning framework implemented in Julia by Deniz Yuret and collaborators. It supports GPU operation and automatic differentiation using dynamic computational graphs for models defined in plain Julia. This document is a tutorial introduction to Knet. Check out the full documentation and Examples for more information. If you need help or would like to request a feature, please consider joining the knet-users mailing list. If you find a bug, please open a GitHub issue. If you would like to contribute to Knet development, check out the knet-dev mailing list and Tips for developers.

https://github.com/denizyuret/Knet.jl

More resources on Julia Programing:

http://online.kitp.ucsb.edu/online/transturb17/gibson/

https://julialang.org/blog/

Feel free to clap and Have fun with Julia. Stay connected.

Programmer’s or Entrepreneur’s life Guide(How to live!)

I just completed Audio version ofSoft Skills: The software developer’s life manualand same after completing of new-skills/books/courses I go with a blog-post, Because I always feel that you can never observer whole book by reading it once and you always need to re-learn things over time because sometimes to learn really new things, you also have to forget  some of  the stuff to learn some of new things as well, And I guess that is one of the most important thing I learned over last few years after my College. So writing a Blog-Post is one of thing I do to Enhance/Validate/Preserve new learnings.

When I first saw John Z. Sonmez on Youtube My first thought was like ” He is so fit, Is he really Software-Engineer” 🙂 How wrong I was.

You can get connected with John here.

 

Ok here comes the Validation of Knowledge about the book mentioned above. You can also use this blog-post to get Excited about book and add it to your Reading List. I am sure you will feel that here is something so common but fresh-about this book, As I am feeling right now that may be I will always read/listen this book at-least once a year.

Be a Specialist: Special Skill is like Brahmanda Astra.

it really does not matter how much you are curios about new technologies but there must be one skill that you will be so good that you can represent yourself at top 1% in your Area/Geo-location/Company/Industry and make sure while doing good amount of work using that skill your produce more Quality than quantity. Alongside that special-skill you must be knowing about sister technologies of that particular skill. There is one more another important thing to remember and that is “Don’t get religious about any particular Technology. “

Learn to Sell yourself High Enough.

Selling yourself only means “Be so great so that nobody can ignore you” but that also does not mean you must write code and build great-softwares by sitting 18-20 hours for day in your basement. You must have off-line as well as on-line good circle of Professional and Merchants in your life. You must have good amount of blog post about the skills you you have learned over time alongside the skills you are learning now due to that you would be so connected with the outside world and most of the people would be knowing that what you really are doing in life.

 Salary Negotiation: It’s always good to negotiate because your Employer know that “It’s good to bargain”

When it comes to salary negotiation it is true that you must be able to know how much you are worth as well as how much more money you need except from your living Expenses. No doubt if you have reached at the level of Salary-Negotiation that means you have already know how much Employer could offer but it never hurts to take that “Moment into bit more negotiation”, Now may be A question could be arise in your mind and that is How really I have to do that? Believe me I am not the guy who can explain it so good as John has descried in the book. So read the book. If you are fan of Audio-Books then Let me know, I will send you Audio version of that book But you will only able to get it free if this is your first book on Audible.com

Investment and Saving: Let your money work for you and you just be sit around to see it is growing. 🙂

So It is really easy to do actually, You just need to understand How much you are able to save and how much you really can do with that saving, Confusing? Money sitting in Bank has no value until or unless you are able to make more of that than Banks do make, and how is that thing possible? There are Wide range of options one can go for like Options-Trading, Futures-Trading, Real State Investment, Making/Investing in one of your own Software Product and many other things. 🙂

For more information about Stocks and Futures please follow the following Articles:

View story at Medium.com

Body and Mental Fitness: The way you look, The way your brain does lot of work for you. 😉

 

There must be really good work-out routine we have to follow/do unless or until you really are not interested growing your brain over the time, Yes that is true only because you have Pump your blood almost everyday and you also have to burn fat, So start running and going to Gym because it is the only way you can not just make your body healthy but also sharp your mind over time.

Diet and Nutrition: You are what you consume.

There is nothing much to say about Diet and eating habits, But make sure you are eating less calories whole day because you have to sit for the most of the day. 🙂

Power of Persistence: Hang in there!

Things work as much as you are staying there for things to make work. Sometimes it takes some time for the results to show up but at some level you have to wait more and let the things fall into the place.

Special One: How money Works.

I never expected John will also talk about great things about Money and Capitalism as well, To understand this you  must have to learn so much stuff and I might not be the person to tell you on this short post that how money works and how you should take it into account. Simple thing about Money is Banks are creating lots of vitalized clusters to generate/initiate the flow of money among different countries and different inside different industries.

Asynchronous recipes in Python

“Concurrency” is not “Parallelism” May be it’s better. If you will not work with DataScience, DataProcessing, Machine-Learning and other operations which are CPU-Intensive you probably will found that you don’t need parallelism but you need concurrency more!

  1. A Simple Example is Training a machine learning model is CPU intensive or You can use GPU.
  2. To Make various Predictions from one model based on many Different Input-Parameters to find out best result You need Concurrency!

There are so Many ways one can hack into Python stuff and do cool Stuff either it is CPU intensive or just a task to do stuff that is good/bad/Better/Best for one user to communicate. One thing you have to believe that Python Does support Multiprocessing as well as Multi-threading

but for various reasons when you are doing CPU intensive Tasks you have to Stay away from using Threading operations in Python. Use Numpy, Cython,Jython or anything you feel, Write C++ code and glue it with Python

The number of threads will usually be equivalent to the number of cores you have. If you have hyperthreading on your processor, than you will be able to double the number of threads used.

Above image is just one Example to understand what actually we are doing. We are processing Chunks and Chuncks of Data. Now the real common scenario is If you are using I/O bound tasks use Threads in Python if you are using CPU bound tasks use Processes in Python.  I have worked with various Python Projects where Performance was issue at some level so at that time I always went to other things like Numpy, Pandas, Cyhton or numba but not Plain-Python.

Let’s come to the point and Point is What are those Recipes I can use:

Using concurrent.futures(futures module is also back-ported into Python2.x):

Suppose you have to call multiple URLs at same time using same Method. That is what actually Concurrency is, Apply same method different operations, We can do it either using ThreadPool or ProcessPool.

# Using Process Pool
from concurrent.futures import ProcessPoolExecutor,as_completed
def health_check1(urls_list):
pool = ProcessPoolExecutor(len(urls_list))
futures = [pool.submit(requests.get,url,verify=False) for url in final_url]
results = [r.result() for r in as_completed(futures)] # when all operations done
return results # a Python list of all results, Here you can also use Numpy as well

Using ThreadPool it is also not different:

# Using Thread Pool
from concurrent.futures import ThreadPoolExecutor,as_completed</code>

def just_func(urls_list):
pool = ThreadPoolExecutor(len(urls_list))
futures = [pool.submit(requests.get,url,verify=False) for url in urls_list]
results = [r.result() for r in as_completed(futures)] # when all operations done
return results # a Python list of all results, Here you can also use Numpy as well

In the above code ‘url_list’ is just list of tasks which are similar and can be processed using same kind of functions.

On the other-side using it with with as context manager is also not different. In this Example I will Use ProcessPoolexecutor’s inbuilt map function.

def just_func(url_list):
   with concurrent.futures.ProcessPoolExecutor(max_workers=len(final_url)) as executor:
        result = executor.map(get_response,final_url)
    return [i for i in result]

Using multiprocessing: (Multiprocessing is also Python-library that can be used for Asynchronous behavior of your code.)

*in Multiprocessing the difference between map and apply_async is only that Map returns results as task list is passed to it on the other-hand apply_async returns results based on results those returned by function.

# Function that run multiple tasks
def get_response(url):
“””returns response for URL ”””
    response = requests.get((url),verify=False)
   return response.text

Now above function is simple enough that is getting one URL and returning response but if have to pass multiple URLs but I want that get request to each URL should be fired at same time then That would be Asynchronous process not multiprocessing because in Multiprocessing Threads/Processes needs to communicate with each other but on the other hand in case of Asynchrounous threads don’t communicate(in Python because Python uses Process based multiprocessing not Thread Based although you can do thread-based multiprocessing in Python but then you are on your OWN 😀 😛 Hail GIL (Mogambo/Hitler)).

So above function will be like this as usual:

from multiprocessing import Pool
pool = Pool(processes=20)
resp_pool = pool.map(get_response,tasks)
URL_list = []
resp_pool = _pool.map(get_response,tasks)
pool.terminate()
pool.join()

Although This is an interesting link one can watch while going into Multiprocessing in Python using Multiprocessing: It is Process-Bases Parallelism.
http://sebastianraschka.com/Articles/2014_multiprocessing.html

Using Gevent: Gevent is a concurrency library based around libev. It provides a clean API for a variety of concurrency and network related tasks.

import gevent
import random

def task(pid):
"""
Some non-deterministic task
"""
    gevent.sleep(random.randint(0,2)*0.001)
    print('Task %s done' % pid)

def asynchronous():
    threads = [gevent.spawn(task, i) for i in xrange(10)]
    gevent.joinall(threads)

    print('Asynchronous:')
asynchronous()

If you have to Call Asynchronously but want to return results in Synchronous Fashion:

import gevent.monkey
gevent.monkey.patch_socket()

import gevent
import urllib2
import simplejson as json

def fetch(pid):
    response = urllib2.urlopen('http://json-time.appspot.com/time.json')
    result = response.read()
    json_result = json.loads(result)
    datetime = json_result['datetime']

    print('Process %s: %s' % (pid, datetime))
    return json_result['datetime']

def asynchronous():
    threads = []
    for i in range(1,10):
        threads.append(gevent.spawn(fetch, i))
    gevent.joinall(threads)

print('Asynchronous:')
asynchronous()

Assigning Jobs in Queue:

import gevent
from gevent.queue import Queue</code>

tasks = Queue()

def worker(n):
    while not tasks.empty():
        task = tasks.get()
        print('Worker %s got task %s' % (n, task))
   gevent.sleep(1)

print('Quitting time!')

def boss():
   for i in xrange(1,25):
      tasks.put_nowait(i)

    gevent.spawn(boss).join()

   gevent.joinall([
gevent.spawn(worker, 'steve'),
gevent.spawn(worker, 'john'),
gevent.spawn(worker, 'nancy'),
])

When you have to manage Different Groups of Asynchronous Tasks:

import gevent
from gevent.pool import Group</code>

def talk(msg):
    for i in xrange(3):
        print(msg)

g1 = gevent.spawn(talk, 'bar')
g2 = gevent.spawn(talk, 'foo')
g3 = gevent.spawn(talk, 'fizz')

group = Group()
group.add(g1)
group.add(g2)
group.join()

group.add(g3)
group.join()

Same As multiprocessing Library you can also use Pool to map various operations:

import gevent
from gevent.pool import Pool</code>

pool = Pool(2)

def hello_from(n):
    print('Size of pool %s' % len(pool))

pool.map(hello_from, xrange(3))

Using Asyncio:

Now let’s talk about concurrency Again! There is already lot of automation is going inside asyncio or Gevent but as programmer we have to understand how we need to break a “One large task into small chuncks of Subtasks so when we will write code we will be able to understand which tasks can work independently.

import time
import asyncio

start = time.time()

def tic():
    return 'at %1.1f seconds' % (time.time() - start)

async def gr1():
# Busy waits for a second, but we don't want to stick around...
    print('gr1 started work: {}'.format(tic()))
    await asyncio.sleep(2)
    print('gr1 ended work: {}'.format(tic()))

async def gr2():
# Busy waits for a second, but we don't want to stick around...
    print('gr2 started work: {}'.format(tic()))
    await asyncio.sleep(2)
    print('gr2 Ended work: {}'.format(tic()))

async def gr3():
    print("Let's do some stuff while the coroutines are blocked, {}".format(tic()))
    await asyncio.sleep(1)
    print("Done!")

ioloop = asyncio.get_event_loop()
tasks = [
ioloop.create_task(gr1()),
ioloop.create_task(gr2()),
ioloop.create_task(gr3())
]
ioloop.run_until_complete(asyncio.wait(tasks))
ioloop.close()

Now in the above code gr1 and gr2 are somehow taking some time to return anything it could any kind of i/o operation so what we can do here is go to the gr3 in using the event_loop and event_loop will run until all three tasks are not completed.

Please have a closer look at await keyword in the above code. It is one of the most important step where you can assume interpreter is shifting from one task to another or you can call it pause for function. If you have worked with yield or yield from in Python2 and Python3 you would be able to understand that this is stateless step for the code.

There is on more library which is aiohttp that is being used to handle blocking Http requests with asyncio.

import time
import asyncio
import aiohttp

URL = 'https://api.github.com/events'
MAX_CLIENTS = 3

async def fetch_async(pid):
    print('Fetch async process {} started'.format(pid))
    start = time.time()
    response = await aiohttp.request('GET', URL)
    return response

async def asynchronous():
    start = time.time()
   tasks = [asyncio.ensure_future(fetch_async(i)) for i in range(1, MAX_CLIENTS +1)]
   await asyncio.wait(tasks)
    print("Process took: {:.2f} seconds".format(time.time() - start))

print('Asynchronous:')
ioloop = asyncio.get_event_loop()
ioloop.run_until_complete(asynchronous())
ioloop.close()

In all the above Examples we have just Scratched the world of concurrency but in real there would be much more to look into because real world problems are more complex and intensive. There are various other options in asyncio like handling exceptions with-in futures, creating future wrappers for normal tasks,Applying timeouts if task is taking more than required time and doing something else instead.

There is lot of inspiration I got while learning about concurrent programming in Python from the following Sources:

https://hackernoon.com/asyncio-for-the-working-python-developer-5c468e6e2e8e
http://www.gevent.org/
https://www.binpress.com/tutorial/simple-python-parallelism/121
http://masnun.com/2016/03/29/python-a-quick-introduction-to-the-concurrent-futures-module.html

%d bloggers like this: